Jul 14 00:44:23.041418 ip-10-0-134-143 systemd[1]: Starting Kubernetes Kubelet... Jul 14 00:44:23.395187 ip-10-0-134-143 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 00:44:23.395187 ip-10-0-134-143 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 00:44:23.395187 ip-10-0-134-143 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 00:44:23.395187 ip-10-0-134-143 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 00:44:23.395187 ip-10-0-134-143 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 00:44:23.396841 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.396777 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 00:44:23.402887 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402868 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:44:23.402887 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402885 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:44:23.402887 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402889 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:44:23.402887 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402892 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402896 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402898 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402901 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402904 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402906 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402909 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402912 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402915 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402918 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402920 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402925 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402929 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402932 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402935 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402938 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402941 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402944 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402946 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:44:23.403072 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402950 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402953 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402957 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402959 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402963 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402966 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402968 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402971 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402974 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402977 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402980 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402982 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402985 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402988 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402990 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402994 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402997 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.402999 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403002 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403005 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:44:23.403562 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403007 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403010 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403013 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403015 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403018 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403020 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403023 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403026 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403028 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403031 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403034 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403036 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403039 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403041 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403044 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403047 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403050 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403052 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403055 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:44:23.404044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403058 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403060 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403063 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403066 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403068 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403071 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403074 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403078 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403084 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403087 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403091 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403093 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403096 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403098 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403101 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403104 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403106 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403109 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403111 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403114 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:44:23.404530 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403117 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403119 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403183 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403188 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403192 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403195 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403198 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403201 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403203 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403206 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403208 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403211 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403213 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403216 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403219 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403221 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403224 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.403226 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404494 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404501 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:44:23.404999 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404505 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404508 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404511 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404514 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404517 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404520 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404523 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404526 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404529 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404531 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404534 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404537 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404539 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404542 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404545 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404548 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404551 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404553 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404556 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404558 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:44:23.405508 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404561 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404563 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404566 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404569 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404572 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404575 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404578 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404580 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404583 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404585 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404587 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404590 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404593 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404596 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404599 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404602 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404606 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404609 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404612 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:44:23.405983 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404614 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404617 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404619 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404622 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404624 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404627 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404629 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404631 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404634 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404641 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404644 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404647 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404650 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404652 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404656 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404660 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404663 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404666 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404669 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:44:23.406529 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404672 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404675 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404678 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404680 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404683 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404686 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404689 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404692 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404694 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404697 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404700 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404702 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404705 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404708 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404711 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404714 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404716 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404719 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404721 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:44:23.406991 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404725 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404729 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404731 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404734 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404738 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404741 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404743 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404746 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404749 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404751 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404754 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404756 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404758 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404761 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404763 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404766 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404768 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404771 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404774 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.404777 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:44:23.407472 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405169 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405176 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405181 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405185 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405189 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405192 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405195 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405198 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405201 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405204 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405207 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405209 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405212 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405214 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405217 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405220 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405225 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405228 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405231 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:44:23.407943 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405234 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405238 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405240 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405243 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405245 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405248 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405251 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405253 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405256 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405259 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405261 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405264 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405267 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405270 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405273 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405275 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405278 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405280 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405283 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405285 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:44:23.408435 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405288 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405290 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405293 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405296 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405298 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405301 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405303 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405308 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405311 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405313 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405316 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405319 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405322 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405324 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405326 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405329 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405331 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405334 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405336 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405339 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:44:23.408902 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405341 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405344 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405346 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405349 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405351 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405354 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405357 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405362 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405365 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405368 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405370 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405373 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405375 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405378 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405380 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405383 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405385 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405388 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405390 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405394 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:44:23.409421 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405397 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405400 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405402 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405406 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405408 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405411 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405414 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405416 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405419 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405422 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405424 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405427 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405429 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405432 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405434 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405437 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405439 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405442 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405444 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.405446 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405512 2557 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 00:44:23.409892 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405519 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405525 2557 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405530 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405537 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405542 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405548 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405554 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405557 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405560 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405563 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405568 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405571 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405575 2557 flags.go:64] FLAG: --cgroup-root="" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405578 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405580 2557 flags.go:64] FLAG: --client-ca-file="" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405584 2557 flags.go:64] FLAG: --cloud-provider="external" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405587 2557 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405592 2557 flags.go:64] FLAG: --cluster-domain="" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405595 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405598 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405601 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405605 2557 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405609 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 00:44:23.410418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405611 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405615 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405618 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405621 2557 flags.go:64] FLAG: --contention-profiling="false" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405624 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405627 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405630 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405633 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405637 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405640 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405643 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405646 2557 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405648 2557 flags.go:64] FLAG: --enable-server="true" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405651 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405656 2557 flags.go:64] FLAG: --event-burst="100" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405659 2557 flags.go:64] FLAG: --event-qps="50" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405662 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405665 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405668 2557 flags.go:64] FLAG: --eviction-hard="" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405672 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405675 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405678 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405681 2557 flags.go:64] FLAG: --eviction-soft="" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405684 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405687 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 00:44:23.410975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405694 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405699 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405702 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405704 2557 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405707 2557 flags.go:64] FLAG: --feature-gates="" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405711 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405714 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405717 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405720 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405723 2557 flags.go:64] FLAG: --healthz-port="10248" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405726 2557 flags.go:64] FLAG: --help="false" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405729 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405732 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405735 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405738 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405741 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405744 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405747 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405750 2557 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405753 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405756 2557 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405758 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405761 2557 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405764 2557 flags.go:64] FLAG: --kube-reserved="" Jul 14 00:44:23.411602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405767 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405770 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405774 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405777 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405780 2557 flags.go:64] FLAG: --lock-file="" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405783 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405786 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405789 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405794 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405799 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405802 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405805 2557 flags.go:64] FLAG: --logging-format="text" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405808 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405811 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405814 2557 flags.go:64] FLAG: --manifest-url="" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405816 2557 flags.go:64] FLAG: --manifest-url-header="" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405821 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405824 2557 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405827 2557 flags.go:64] FLAG: --max-pods="110" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405830 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405833 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405836 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405839 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405842 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405845 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 00:44:23.412245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405848 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405855 2557 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405858 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405861 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405863 2557 flags.go:64] FLAG: --pod-cidr="" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405866 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405871 2557 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405874 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405877 2557 flags.go:64] FLAG: --pods-per-core="0" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405881 2557 flags.go:64] FLAG: --port="10250" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405885 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405888 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d699137ee87e5d9a" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405891 2557 flags.go:64] FLAG: --qos-reserved="" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405894 2557 flags.go:64] FLAG: --read-only-port="10255" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405897 2557 flags.go:64] FLAG: --register-node="true" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405900 2557 flags.go:64] FLAG: --register-with-taints="" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405903 2557 flags.go:64] FLAG: --registry-burst="10" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405912 2557 flags.go:64] FLAG: --registry-qps="5" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405915 2557 flags.go:64] FLAG: --reserved-cpus="" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405918 2557 flags.go:64] FLAG: --reserved-memory="" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405922 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405925 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405928 2557 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405931 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405933 2557 flags.go:64] FLAG: --runonce="false" Jul 14 00:44:23.412873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405936 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405939 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405942 2557 flags.go:64] FLAG: --seccomp-default="false" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405945 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405948 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405951 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405953 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405956 2557 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405959 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405962 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405965 2557 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405968 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405971 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405974 2557 flags.go:64] FLAG: --system-cgroups="" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405976 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405981 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405986 2557 flags.go:64] FLAG: --tls-cert-file="" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405989 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405992 2557 flags.go:64] FLAG: --tls-min-version="" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405995 2557 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.405998 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.406001 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.406004 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.406007 2557 flags.go:64] FLAG: --v="2" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.406012 2557 flags.go:64] FLAG: --version="false" Jul 14 00:44:23.413480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.406017 2557 flags.go:64] FLAG: --vmodule="" Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.406020 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.406024 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406173 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406177 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406182 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406186 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406189 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406192 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406195 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406198 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406200 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406203 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406206 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406208 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406211 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406214 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406216 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406219 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406221 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:44:23.414086 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406224 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406226 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406229 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406232 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406235 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406238 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406241 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406244 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406246 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406249 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406252 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406255 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406258 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406261 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406263 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406266 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406269 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406271 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406274 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:44:23.414584 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406276 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406279 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406283 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406286 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406289 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406292 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406295 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406297 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406300 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406303 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406305 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406308 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406310 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406313 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406315 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406318 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406320 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406323 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406326 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:44:23.415044 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406328 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406331 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406333 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406336 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406339 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406342 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406345 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406348 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406351 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406354 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406356 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406359 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406361 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406364 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406367 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406369 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406372 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406375 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406377 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406379 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:44:23.415535 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406382 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406385 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406387 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406390 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406393 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406395 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406398 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406400 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406403 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406406 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406409 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406413 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406416 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406419 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406421 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406424 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406426 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406430 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406433 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:44:23.416034 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406435 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406438 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406441 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406444 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.406447 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.407039 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.413142 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.413173 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.413918 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.413926 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.413981 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.413988 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.413992 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.413995 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:44:23.416503 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.413998 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414001 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414003 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414006 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414009 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414011 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414014 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414017 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414019 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414022 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414024 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414027 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414030 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414032 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414035 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414037 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414040 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414042 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414045 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414048 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:44:23.416849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414050 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414053 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414055 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414058 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414060 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414064 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414067 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414069 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414072 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414075 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414077 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414080 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414082 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414085 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414088 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414090 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414093 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414096 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414098 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414101 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:44:23.417347 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414104 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414106 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414109 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414111 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414114 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414117 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414119 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414122 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414124 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414127 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414130 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414134 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414138 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414140 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414143 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414158 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414163 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414166 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414176 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414179 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:44:23.417849 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414182 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414185 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414187 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414190 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414193 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414195 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414198 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414201 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414203 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414206 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414208 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414211 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414214 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414216 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414219 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414221 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414224 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414226 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414230 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:44:23.418416 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414234 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414237 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414240 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414242 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414245 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414248 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414251 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414253 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414256 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414258 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414261 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414263 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414266 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414269 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414272 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:44:23.418884 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414274 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.414279 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414385 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414390 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414393 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414396 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414399 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414402 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414406 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414408 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414411 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414413 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414416 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414419 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 00:44:23.419271 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414422 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414424 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414427 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414429 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414432 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414435 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414437 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414440 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414442 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414445 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414447 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414450 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414452 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414455 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414457 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414460 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414462 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414465 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414467 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414470 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 00:44:23.419614 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414472 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414475 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414477 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414480 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414482 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414486 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414489 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414492 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414495 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414498 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414501 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414503 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414506 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414509 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414512 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414514 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414517 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414519 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414522 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 00:44:23.420093 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414524 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414527 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414530 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414532 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414535 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414537 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414540 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414542 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414545 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414547 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414550 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414553 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414555 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414558 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414560 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414562 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414565 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414567 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414570 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414572 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 00:44:23.420574 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414575 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414578 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414580 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414583 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414585 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414588 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414591 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414595 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414598 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414601 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414603 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414606 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414608 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414611 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414613 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414616 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414618 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414621 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414623 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 00:44:23.421042 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414626 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414628 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414631 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414633 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414636 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414638 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414641 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414643 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:23.414646 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.414651 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.414762 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.416944 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.419011 2557 server.go:1019] "Starting client certificate rotation" Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.419107 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 00:44:23.421526 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.419846 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 00:44:23.421867 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.421360 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 00:44:23.425968 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.425949 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 00:44:23.439794 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.439772 2557 log.go:25] "Validated CRI v1 runtime API" Jul 14 00:44:23.441903 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.441891 2557 log.go:25] "Validated CRI v1 image API" Jul 14 00:44:23.442704 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.442687 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 00:44:23.447633 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.447610 2557 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aff51e2f-5a3c-49a6-b829-6c9bd158efd9:/dev/nvme0n1p4 fe077554-b861-4000-aa97-1038e051d22c:/dev/nvme0n1p3] Jul 14 00:44:23.447699 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.447629 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 00:44:23.451912 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.451810 2557 manager.go:216] Machine: {Timestamp:2026-07-14 00:44:23.451353173 +0000 UTC m=+0.302720534 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100729 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c8abfd05694ed0f1f8c916ae69044 SystemUUID:ec2c8abf-d056-94ed-0f1f-8c916ae69044 BootID:74d7a46a-6994-4308-8925-3bd9eef869c5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d7:fa:bc:db:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d7:fa:bc:db:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:21:00:1d:31:de Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 00:44:23.451912 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.451909 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 00:44:23.452039 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.451994 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 00:44:23.452736 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.452721 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 00:44:23.455129 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.455101 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 00:44:23.455275 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.455132 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-143.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 00:44:23.455755 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.455746 2557 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 00:44:23.455788 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.455757 2557 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 00:44:23.455788 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.455764 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 00:44:23.456412 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.456402 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 00:44:23.457524 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.457515 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 00:44:23.460912 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.460901 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 14 00:44:23.461027 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.461018 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 00:44:23.461061 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.461048 2557 kubelet.go:486] "Attempting to sync node with API server" Jul 14 00:44:23.461088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.461061 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 00:44:23.461088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.461076 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 00:44:23.461088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.461084 2557 kubelet.go:398] "Adding apiserver pod source" Jul 14 00:44:23.461246 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.461091 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 00:44:23.461423 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.461412 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 00:44:23.461457 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.461444 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 00:44:23.463333 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.463311 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 00:44:23.467228 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.467211 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 00:44:23.467307 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.467264 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 00:44:23.467906 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.467887 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 00:44:23.468180 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468168 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 00:44:23.468218 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468182 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 00:44:23.468218 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468188 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 00:44:23.468218 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468197 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 00:44:23.468218 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468203 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 00:44:23.468218 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468211 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 00:44:23.468218 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468216 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 00:44:23.468411 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468224 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 00:44:23.468411 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468230 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 00:44:23.468411 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468239 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 00:44:23.468411 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468244 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 00:44:23.468411 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468268 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 00:44:23.468411 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.468273 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 00:44:23.471452 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.471436 2557 server.go:1268] "Started kubelet" Jul 14 00:44:23.471525 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.471483 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 00:44:23.471656 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.471609 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 00:44:23.471751 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.471732 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 00:44:23.472290 ip-10-0-134-143 systemd[1]: Started Kubernetes Kubelet. Jul 14 00:44:23.473600 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.473582 2557 server.go:310] "Adding debug handlers to kubelet server" Jul 14 00:44:23.473740 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.473724 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-143.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 00:44:23.473806 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.473741 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 00:44:23.474480 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.474443 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 00:44:23.474480 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.474469 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-143.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 00:44:23.477495 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.477473 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 00:44:23.481571 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.481557 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 00:44:23.481639 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.481558 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 00:44:23.481700 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.481676 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:23.482273 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.482252 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 00:44:23.482368 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.482352 2557 factory.go:55] Registering systemd factory Jul 14 00:44:23.482431 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.482373 2557 factory.go:223] Registration of the systemd container factory successfully Jul 14 00:44:23.482680 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.482666 2557 factory.go:153] Registering CRI-O factory Jul 14 00:44:23.482741 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.482685 2557 factory.go:223] Registration of the crio container factory successfully Jul 14 00:44:23.482793 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.482747 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 00:44:23.482793 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.482771 2557 factory.go:103] Registering Raw factory Jul 14 00:44:23.482793 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.482786 2557 manager.go:1196] Started watching for new ooms in manager Jul 14 00:44:23.483188 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.483174 2557 manager.go:349] Starting recovery of all containers Jul 14 00:44:23.483323 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.483306 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 00:44:23.483377 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.483345 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 00:44:23.483552 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.483538 2557 reconstruct.go:97] "Volume reconstruction finished" Jul 14 00:44:23.483552 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.483550 2557 reconciler.go:29] "Reconciler: start to sync state" Jul 14 00:44:23.484383 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.483446 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-143.ec2.internal.18c201542f1bd3d1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-143.ec2.internal,UID:ip-10-0-134-143.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-143.ec2.internal,},FirstTimestamp:2026-07-14 00:44:23.471371217 +0000 UTC m=+0.322738578,LastTimestamp:2026-07-14 00:44:23.471371217 +0000 UTC m=+0.322738578,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-143.ec2.internal,}" Jul 14 00:44:23.487476 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.487450 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-143.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 00:44:23.487768 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.487745 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 00:44:23.495139 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.495003 2557 manager.go:354] Recovery completed Jul 14 00:44:23.496679 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.496663 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zj5ww" Jul 14 00:44:23.499097 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.499085 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:44:23.500910 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.500896 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:44:23.500974 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.500923 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:44:23.500974 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.500932 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:44:23.501419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.501406 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 00:44:23.501419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.501418 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:44:23.501535 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.501420 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 00:44:23.501535 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.501455 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 14 00:44:23.502141 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.502126 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:44:23.502269 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.502233 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:44:23.502269 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.502247 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:44:23.502787 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.502773 2557 policy_none.go:49] "None policy: Start" Jul 14 00:44:23.502850 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.502793 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 00:44:23.502850 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.502807 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 00:44:23.503741 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.503729 2557 policy_none.go:47] "Start" Jul 14 00:44:23.506922 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.506828 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-143.ec2.internal.18c2015430de8b2f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-143.ec2.internal,UID:ip-10-0-134-143.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-143.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-143.ec2.internal,},FirstTimestamp:2026-07-14 00:44:23.500909359 +0000 UTC m=+0.352276720,LastTimestamp:2026-07-14 00:44:23.500909359 +0000 UTC m=+0.352276720,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-143.ec2.internal,}" Jul 14 00:44:23.507024 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.507004 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zj5ww" Jul 14 00:44:23.543683 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.543668 2557 manager.go:337] "Starting Device Plugin manager" Jul 14 00:44:23.543743 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.543694 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 00:44:23.543743 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.543703 2557 server.go:87] "Starting device plugin registration server" Jul 14 00:44:23.543915 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.543904 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 00:44:23.543985 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.543916 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 00:44:23.544042 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.544019 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 00:44:23.544097 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.544084 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 00:44:23.544097 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.544091 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 00:44:23.544555 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.544532 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 00:44:23.544620 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.544584 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:23.644888 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.644867 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:44:23.645839 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.645797 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:44:23.645839 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.645825 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:44:23.645839 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.645835 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:44:23.645987 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.645858 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.654510 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.654478 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.654600 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.654528 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-143.ec2.internal\": node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:23.669667 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.669647 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 00:44:23.670129 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.670111 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:23.670921 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.670902 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 00:44:23.671439 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.671421 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 00:44:23.671530 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.671457 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 00:44:23.671530 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.671493 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 00:44:23.673702 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.673686 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:44:23.770674 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.770647 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:23.771838 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.771820 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-143.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal"] Jul 14 00:44:23.772379 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.772365 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:44:23.773175 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.773151 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:44:23.773262 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.773184 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:44:23.773262 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.773196 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:44:23.775244 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.774899 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:44:23.775244 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.775093 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.775244 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.775129 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:44:23.776073 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.776056 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:44:23.776073 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.776069 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:44:23.776285 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.776087 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:44:23.776285 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.776094 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:44:23.776285 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.776105 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:44:23.776285 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.776106 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:44:23.777407 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.777388 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.777497 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.777421 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 00:44:23.778119 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.778101 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientMemory" Jul 14 00:44:23.778243 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.778133 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 00:44:23.778243 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.778146 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeHasSufficientPID" Jul 14 00:44:23.785986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.785966 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68f0c074f7c50b5b863d7294622d4fc6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-143.ec2.internal\" (UID: \"68f0c074f7c50b5b863d7294622d4fc6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.786081 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.785998 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/effd29edc9dfe1c426b92a89b21d0dbd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal\" (UID: \"effd29edc9dfe1c426b92a89b21d0dbd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.786081 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.786026 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/effd29edc9dfe1c426b92a89b21d0dbd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal\" (UID: \"effd29edc9dfe1c426b92a89b21d0dbd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.797784 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.797760 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-143.ec2.internal\" not found" node="ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.802364 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.802338 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-143.ec2.internal\" not found" node="ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.871001 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.870986 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:23.886213 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.886181 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/effd29edc9dfe1c426b92a89b21d0dbd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal\" (UID: \"effd29edc9dfe1c426b92a89b21d0dbd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.886319 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.886223 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/effd29edc9dfe1c426b92a89b21d0dbd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal\" (UID: \"effd29edc9dfe1c426b92a89b21d0dbd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.886319 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.886241 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68f0c074f7c50b5b863d7294622d4fc6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-143.ec2.internal\" (UID: \"68f0c074f7c50b5b863d7294622d4fc6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.886319 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.886247 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/effd29edc9dfe1c426b92a89b21d0dbd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal\" (UID: \"effd29edc9dfe1c426b92a89b21d0dbd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.886319 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.886282 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68f0c074f7c50b5b863d7294622d4fc6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-143.ec2.internal\" (UID: \"68f0c074f7c50b5b863d7294622d4fc6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.886319 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:23.886301 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/effd29edc9dfe1c426b92a89b21d0dbd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal\" (UID: \"effd29edc9dfe1c426b92a89b21d0dbd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" Jul 14 00:44:23.971937 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:23.971891 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:24.078531 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:24.078507 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:24.100589 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.100564 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-143.ec2.internal" Jul 14 00:44:24.106146 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.106130 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" Jul 14 00:44:24.179905 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:24.179879 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:24.281483 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:24.281436 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:24.382602 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:24.382576 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:24.418632 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.418609 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 00:44:24.419166 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.418725 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 00:44:24.482925 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:24.482901 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-143.ec2.internal\" not found" Jul 14 00:44:24.511742 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.511695 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 00:39:24 +0000 UTC" deadline="2028-01-06 14:59:51.155021688 +0000 UTC" Jul 14 00:44:24.511742 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.511736 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12998h15m26.643288378s" Jul 14 00:44:24.532108 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.532067 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:44:24.582272 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.582244 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-143.ec2.internal" Jul 14 00:44:24.592417 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.592398 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 00:44:24.593975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.593954 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" Jul 14 00:44:24.602947 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.602925 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 00:44:24.723349 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:24.723316 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeffd29edc9dfe1c426b92a89b21d0dbd.slice/crio-7273b65b9f467cbebbafdc42e5af0dc60a3dd73921e3eef2a956a9e37a8cdd87 WatchSource:0}: Error finding container 7273b65b9f467cbebbafdc42e5af0dc60a3dd73921e3eef2a956a9e37a8cdd87: Status 404 returned error can't find the container with id 7273b65b9f467cbebbafdc42e5af0dc60a3dd73921e3eef2a956a9e37a8cdd87 Jul 14 00:44:24.723615 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:24.723596 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f0c074f7c50b5b863d7294622d4fc6.slice/crio-0562b08e7a8b5d356ef451372c5d87122beac84ca39d3f522e13d196e6141ced WatchSource:0}: Error finding container 0562b08e7a8b5d356ef451372c5d87122beac84ca39d3f522e13d196e6141ced: Status 404 returned error can't find the container with id 0562b08e7a8b5d356ef451372c5d87122beac84ca39d3f522e13d196e6141ced Jul 14 00:44:24.729774 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.729757 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 00:44:24.969131 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:24.969105 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:44:25.071576 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.071547 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:44:25.462980 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.462956 2557 apiserver.go:52] "Watching apiserver" Jul 14 00:44:25.467900 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.467876 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 00:44:25.470483 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.470451 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-tzdlx","openshift-network-operator/iptables-alerter-hvmtb","kube-system/global-pull-secret-syncer-nzt74","openshift-dns/node-resolver-7xct5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal","openshift-multus/multus-frmbq","openshift-multus/network-metrics-daemon-txzdm","openshift-ovn-kubernetes/ovnkube-node-v9t68","kube-system/konnectivity-agent-9dwkf","kube-system/kube-apiserver-proxy-ip-10-0-134-143.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76","openshift-cluster-node-tuning-operator/tuned-l9mwk","openshift-image-registry/node-ca-qqpzf","openshift-multus/multus-additional-cni-plugins-njjqg"] Jul 14 00:44:25.472578 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.472436 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:25.472578 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:25.472540 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:25.474086 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.474064 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.475822 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.475643 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:25.475822 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:25.475760 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:25.476008 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.475921 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cvglv\"" Jul 14 00:44:25.476008 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.475925 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 00:44:25.476123 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.476027 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 00:44:25.476190 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.476129 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 00:44:25.477107 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.477078 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7xct5" Jul 14 00:44:25.478971 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.478794 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.478971 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.478886 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 00:44:25.478971 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.478937 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 00:44:25.480128 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.480067 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-t9jtq\"" Jul 14 00:44:25.480398 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.480372 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.481295 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.481271 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 00:44:25.482761 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.482738 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 00:44:25.483245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.483061 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 00:44:25.483370 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.483351 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 00:44:25.483648 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.483625 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 00:44:25.483725 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.483694 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6zfm6\"" Jul 14 00:44:25.483884 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.483864 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 00:44:25.483991 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.483939 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 00:44:25.484159 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.484141 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:25.484598 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.484446 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-76jkd\"" Jul 14 00:44:25.485322 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.485112 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qqpzf" Jul 14 00:44:25.486574 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.486384 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 00:44:25.486574 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.486557 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 00:44:25.486743 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.486576 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lqtgc\"" Jul 14 00:44:25.488134 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.487380 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 00:44:25.488134 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.487414 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 00:44:25.488134 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.487668 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 00:44:25.488134 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.487972 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j7ppb\"" Jul 14 00:44:25.488134 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.488075 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hvmtb" Jul 14 00:44:25.489189 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.489130 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.490793 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.489982 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 00:44:25.490793 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.490032 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 00:44:25.490793 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.490116 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zk8l5\"" Jul 14 00:44:25.490793 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.490179 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 00:44:25.491178 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.491126 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 00:44:25.491783 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.491634 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 00:44:25.491933 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.491915 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:25.492010 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.491935 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 00:44:25.492010 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.491987 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 00:44:25.492124 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.492014 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tgw6v\"" Jul 14 00:44:25.492124 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:25.492100 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:25.492288 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.492268 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 00:44:25.492517 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.492501 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 00:44:25.493101 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493080 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.493511 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493482 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/afc9e28a-2764-4e91-884b-ead5986764cc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.493607 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493542 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-etc-kubernetes\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.493607 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493575 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxc5z\" (UniqueName: \"kubernetes.io/projected/cea48c24-3755-41c7-b30c-5d60aa682341-kube-api-access-dxc5z\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.493607 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493600 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9529\" (UniqueName: \"kubernetes.io/projected/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-kube-api-access-k9529\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:25.493757 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493624 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-socket-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.493757 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493676 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8dacd2e-6336-4956-aa9f-e9a0964534d0-tmp-dir\") pod \"node-resolver-7xct5\" (UID: \"c8dacd2e-6336-4956-aa9f-e9a0964534d0\") " pod="openshift-dns/node-resolver-7xct5" Jul 14 00:44:25.493757 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493719 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-kubelet-config\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:25.493910 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493748 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-cnibin\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.493910 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493802 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/afc9e28a-2764-4e91-884b-ead5986764cc-cni-binary-copy\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.493910 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493828 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.493910 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493868 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-cnibin\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494117 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493939 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-os-release\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494117 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.493978 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cea48c24-3755-41c7-b30c-5d60aa682341-cni-binary-copy\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494117 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494006 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.494117 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494048 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-os-release\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.494117 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494096 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8km4d\" (UniqueName: \"kubernetes.io/projected/afc9e28a-2764-4e91-884b-ead5986764cc-kube-api-access-8km4d\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.494433 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494130 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-system-cni-dir\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494433 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494158 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-multus-socket-dir-parent\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494433 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494201 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-run-netns\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494433 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494224 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-hostroot\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494433 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494247 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-multus-conf-dir\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494433 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494271 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-dbus\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:25.494433 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494314 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-registration-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.494433 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494339 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-device-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.494433 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494385 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-sys-fs\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.494433 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494409 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:25.494433 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494433 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-system-cni-dir\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494457 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-run-k8s-cni-cncf-io\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494493 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cea48c24-3755-41c7-b30c-5d60aa682341-multus-daemon-config\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494536 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8dacd2e-6336-4956-aa9f-e9a0964534d0-hosts-file\") pod \"node-resolver-7xct5\" (UID: \"c8dacd2e-6336-4956-aa9f-e9a0964534d0\") " pod="openshift-dns/node-resolver-7xct5" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494568 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-multus-cni-dir\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494592 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/427427dc-5248-4fdb-b49f-1309f98f2320-host\") pod \"node-ca-qqpzf\" (UID: \"427427dc-5248-4fdb-b49f-1309f98f2320\") " pod="openshift-image-registry/node-ca-qqpzf" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494616 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwvfd\" (UniqueName: \"kubernetes.io/projected/427427dc-5248-4fdb-b49f-1309f98f2320-kube-api-access-rwvfd\") pod \"node-ca-qqpzf\" (UID: \"427427dc-5248-4fdb-b49f-1309f98f2320\") " pod="openshift-image-registry/node-ca-qqpzf" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494641 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-etc-selinux\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494677 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxlnk\" (UniqueName: \"kubernetes.io/projected/2be9a63e-5667-4342-a081-2554ff60db8a-kube-api-access-wxlnk\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494707 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/afc9e28a-2764-4e91-884b-ead5986764cc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494735 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6cjt\" (UniqueName: \"kubernetes.io/projected/c8dacd2e-6336-4956-aa9f-e9a0964534d0-kube-api-access-f6cjt\") pod \"node-resolver-7xct5\" (UID: \"c8dacd2e-6336-4956-aa9f-e9a0964534d0\") " pod="openshift-dns/node-resolver-7xct5" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494772 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-var-lib-cni-bin\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494817 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-var-lib-cni-multus\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494843 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/427427dc-5248-4fdb-b49f-1309f98f2320-serviceca\") pod \"node-ca-qqpzf\" (UID: \"427427dc-5248-4fdb-b49f-1309f98f2320\") " pod="openshift-image-registry/node-ca-qqpzf" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494868 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494895 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-var-lib-kubelet\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.494986 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494943 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-run-multus-certs\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.495824 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494971 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b4febc6f-5f93-4d99-bf2c-1a4211db67f8-agent-certs\") pod \"konnectivity-agent-9dwkf\" (UID: \"b4febc6f-5f93-4d99-bf2c-1a4211db67f8\") " pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:25.495824 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.494996 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b4febc6f-5f93-4d99-bf2c-1a4211db67f8-konnectivity-ca\") pod \"konnectivity-agent-9dwkf\" (UID: \"b4febc6f-5f93-4d99-bf2c-1a4211db67f8\") " pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:25.495824 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.495118 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2kftk\"" Jul 14 00:44:25.495824 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.495127 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 00:44:25.495824 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.495124 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 00:44:25.495824 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.495390 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 00:44:25.525044 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.525026 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l47p4" Jul 14 00:44:25.531304 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.531281 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l47p4" Jul 14 00:44:25.584395 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.584373 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 00:44:25.596073 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596042 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srgfd\" (UniqueName: \"kubernetes.io/projected/d146a5e0-9ae6-4976-b3fb-26a5f033c39d-kube-api-access-srgfd\") pod \"iptables-alerter-hvmtb\" (UID: \"d146a5e0-9ae6-4976-b3fb-26a5f033c39d\") " pod="openshift-network-operator/iptables-alerter-hvmtb" Jul 14 00:44:25.596200 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596084 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-run-openvswitch\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.596200 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596112 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-node-log\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.596200 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596137 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2bae43db-11b2-47ff-951c-604be5df51cd-env-overrides\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.596200 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596180 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:25.596419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596208 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-run-multus-certs\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.596419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596232 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-systemd\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.596419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596255 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-run\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.596419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596282 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-lib-modules\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.596419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596306 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-var-lib-kubelet\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.596419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596327 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-tuned\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.596419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596352 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b4febc6f-5f93-4d99-bf2c-1a4211db67f8-konnectivity-ca\") pod \"konnectivity-agent-9dwkf\" (UID: \"b4febc6f-5f93-4d99-bf2c-1a4211db67f8\") " pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:25.596419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596383 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-etc-kubernetes\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.596419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596407 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d146a5e0-9ae6-4976-b3fb-26a5f033c39d-iptables-alerter-script\") pod \"iptables-alerter-hvmtb\" (UID: \"d146a5e0-9ae6-4976-b3fb-26a5f033c39d\") " pod="openshift-network-operator/iptables-alerter-hvmtb" Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596430 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-kubernetes\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596455 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9529\" (UniqueName: \"kubernetes.io/projected/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-kube-api-access-k9529\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596482 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-run-netns\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596504 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-sysctl-conf\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596530 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/afc9e28a-2764-4e91-884b-ead5986764cc-cni-binary-copy\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596555 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-cnibin\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596580 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cea48c24-3755-41c7-b30c-5d60aa682341-cni-binary-copy\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:25.596585 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596531 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-run-multus-certs\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596655 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-cnibin\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.596696 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-etc-kubernetes\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.596849 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:25.596706 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs podName:4f92c479-1507-4bd3-a7a3-0bcdb89a049f nodeName:}" failed. No retries permitted until 2026-07-14 00:44:26.096667291 +0000 UTC m=+2.948034672 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs") pod "network-metrics-daemon-txzdm" (UID: "4f92c479-1507-4bd3-a7a3-0bcdb89a049f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:25.597537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597273 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/afc9e28a-2764-4e91-884b-ead5986764cc-cni-binary-copy\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.597537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597330 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-multus-conf-dir\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.597537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597377 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-var-lib-openvswitch\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.597537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597387 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-multus-conf-dir\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.597537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597399 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cea48c24-3755-41c7-b30c-5d60aa682341-cni-binary-copy\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.597537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597430 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b65z2\" (UniqueName: \"kubernetes.io/projected/2bae43db-11b2-47ff-951c-604be5df51cd-kube-api-access-b65z2\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.597537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597460 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-sysctl-d\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.597537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597501 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8km4d\" (UniqueName: \"kubernetes.io/projected/afc9e28a-2764-4e91-884b-ead5986764cc-kube-api-access-8km4d\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.597537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597527 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d146a5e0-9ae6-4976-b3fb-26a5f033c39d-host-slash\") pod \"iptables-alerter-hvmtb\" (UID: \"d146a5e0-9ae6-4976-b3fb-26a5f033c39d\") " pod="openshift-network-operator/iptables-alerter-hvmtb" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597568 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2bae43db-11b2-47ff-951c-604be5df51cd-ovnkube-config\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597641 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-dbus\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597676 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-device-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597704 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-system-cni-dir\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597731 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-run-k8s-cni-cncf-io\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597755 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cea48c24-3755-41c7-b30c-5d60aa682341-multus-daemon-config\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597782 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-cni-netd\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597796 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-system-cni-dir\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597809 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597836 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-sys\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597846 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-dbus\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597860 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-device-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597892 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-run-k8s-cni-cncf-io\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597862 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8dacd2e-6336-4956-aa9f-e9a0964534d0-hosts-file\") pod \"node-resolver-7xct5\" (UID: \"c8dacd2e-6336-4956-aa9f-e9a0964534d0\") " pod="openshift-dns/node-resolver-7xct5" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597918 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c8dacd2e-6336-4956-aa9f-e9a0964534d0-hosts-file\") pod \"node-resolver-7xct5\" (UID: \"c8dacd2e-6336-4956-aa9f-e9a0964534d0\") " pod="openshift-dns/node-resolver-7xct5" Jul 14 00:44:25.597978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597957 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-multus-cni-dir\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.597989 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnlj4\" (UniqueName: \"kubernetes.io/projected/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-kube-api-access-nnlj4\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598024 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-multus-cni-dir\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598033 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b4febc6f-5f93-4d99-bf2c-1a4211db67f8-konnectivity-ca\") pod \"konnectivity-agent-9dwkf\" (UID: \"b4febc6f-5f93-4d99-bf2c-1a4211db67f8\") " pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598065 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/afc9e28a-2764-4e91-884b-ead5986764cc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598094 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-var-lib-cni-multus\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598186 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/427427dc-5248-4fdb-b49f-1309f98f2320-serviceca\") pod \"node-ca-qqpzf\" (UID: \"427427dc-5248-4fdb-b49f-1309f98f2320\") " pod="openshift-image-registry/node-ca-qqpzf" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598273 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-var-lib-cni-multus\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598301 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-etc-openvswitch\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598357 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-var-lib-kubelet\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598386 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598357 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cea48c24-3755-41c7-b30c-5d60aa682341-multus-daemon-config\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598407 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-var-lib-kubelet\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598413 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-systemd-units\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598442 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b4febc6f-5f93-4d99-bf2c-1a4211db67f8-agent-certs\") pod \"konnectivity-agent-9dwkf\" (UID: \"b4febc6f-5f93-4d99-bf2c-1a4211db67f8\") " pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598494 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/afc9e28a-2764-4e91-884b-ead5986764cc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598615 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxc5z\" (UniqueName: \"kubernetes.io/projected/cea48c24-3755-41c7-b30c-5d60aa682341-kube-api-access-dxc5z\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.598712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598644 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/afc9e28a-2764-4e91-884b-ead5986764cc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598685 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-kubelet\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598719 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-run-ovn\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598755 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-socket-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598782 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598795 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8dacd2e-6336-4956-aa9f-e9a0964534d0-tmp-dir\") pod \"node-resolver-7xct5\" (UID: \"c8dacd2e-6336-4956-aa9f-e9a0964534d0\") " pod="openshift-dns/node-resolver-7xct5" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598822 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-slash\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598855 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-cni-bin\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598898 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-kubelet-config\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598924 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-cnibin\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.598985 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-kubelet-config\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599001 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/afc9e28a-2764-4e91-884b-ead5986764cc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599017 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599055 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-os-release\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599096 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-cnibin\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599111 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8dacd2e-6336-4956-aa9f-e9a0964534d0-tmp-dir\") pod \"node-resolver-7xct5\" (UID: \"c8dacd2e-6336-4956-aa9f-e9a0964534d0\") " pod="openshift-dns/node-resolver-7xct5" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599148 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-hostroot\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599200 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwvfd\" (UniqueName: \"kubernetes.io/projected/427427dc-5248-4fdb-b49f-1309f98f2320-kube-api-access-rwvfd\") pod \"node-ca-qqpzf\" (UID: \"427427dc-5248-4fdb-b49f-1309f98f2320\") " pod="openshift-image-registry/node-ca-qqpzf" Jul 14 00:44:25.599670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599216 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-socket-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599234 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-sysconfig\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599283 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-hostroot\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599282 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599318 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-run-systemd\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599347 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599370 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/427427dc-5248-4fdb-b49f-1309f98f2320-serviceca\") pod \"node-ca-qqpzf\" (UID: \"427427dc-5248-4fdb-b49f-1309f98f2320\") " pod="openshift-image-registry/node-ca-qqpzf" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599375 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-os-release\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599405 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599417 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-system-cni-dir\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599425 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-os-release\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599443 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-multus-socket-dir-parent\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599460 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-system-cni-dir\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599467 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-run-netns\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599442 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/afc9e28a-2764-4e91-884b-ead5986764cc-os-release\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599490 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-multus-socket-dir-parent\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599500 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-log-socket\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.600568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599518 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-run-netns\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599526 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2bae43db-11b2-47ff-951c-604be5df51cd-ovnkube-script-lib\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599554 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-registration-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599577 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-sys-fs\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599603 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599616 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-registration-dir\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599628 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/427427dc-5248-4fdb-b49f-1309f98f2320-host\") pod \"node-ca-qqpzf\" (UID: \"427427dc-5248-4fdb-b49f-1309f98f2320\") " pod="openshift-image-registry/node-ca-qqpzf" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599645 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-sys-fs\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599669 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/427427dc-5248-4fdb-b49f-1309f98f2320-host\") pod \"node-ca-qqpzf\" (UID: \"427427dc-5248-4fdb-b49f-1309f98f2320\") " pod="openshift-image-registry/node-ca-qqpzf" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:25.599698 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:25.599745 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret podName:5d75eb52-5424-4c0e-a6bc-1fe4d642587a nodeName:}" failed. No retries permitted until 2026-07-14 00:44:26.099728092 +0000 UTC m=+2.951095457 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret") pod "global-pull-secret-syncer-nzt74" (UID: "5d75eb52-5424-4c0e-a6bc-1fe4d642587a") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599765 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2bae43db-11b2-47ff-951c-604be5df51cd-ovn-node-metrics-cert\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599794 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmjsd\" (UniqueName: \"kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd\") pod \"network-check-target-tzdlx\" (UID: \"38610aec-8e4e-492e-8f63-1e34067176d0\") " pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599829 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-modprobe-d\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599859 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-host\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599884 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-tmp\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.601458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599912 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-etc-selinux\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.602267 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599938 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxlnk\" (UniqueName: \"kubernetes.io/projected/2be9a63e-5667-4342-a081-2554ff60db8a-kube-api-access-wxlnk\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.602267 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.599959 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6cjt\" (UniqueName: \"kubernetes.io/projected/c8dacd2e-6336-4956-aa9f-e9a0964534d0-kube-api-access-f6cjt\") pod \"node-resolver-7xct5\" (UID: \"c8dacd2e-6336-4956-aa9f-e9a0964534d0\") " pod="openshift-dns/node-resolver-7xct5" Jul 14 00:44:25.602267 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.600004 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2be9a63e-5667-4342-a081-2554ff60db8a-etc-selinux\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.602267 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.600119 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-var-lib-cni-bin\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.602267 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.600228 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cea48c24-3755-41c7-b30c-5d60aa682341-host-var-lib-cni-bin\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.604133 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.604080 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b4febc6f-5f93-4d99-bf2c-1a4211db67f8-agent-certs\") pod \"konnectivity-agent-9dwkf\" (UID: \"b4febc6f-5f93-4d99-bf2c-1a4211db67f8\") " pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:25.611841 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.611781 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6cjt\" (UniqueName: \"kubernetes.io/projected/c8dacd2e-6336-4956-aa9f-e9a0964534d0-kube-api-access-f6cjt\") pod \"node-resolver-7xct5\" (UID: \"c8dacd2e-6336-4956-aa9f-e9a0964534d0\") " pod="openshift-dns/node-resolver-7xct5" Jul 14 00:44:25.612902 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.612417 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8km4d\" (UniqueName: \"kubernetes.io/projected/afc9e28a-2764-4e91-884b-ead5986764cc-kube-api-access-8km4d\") pod \"multus-additional-cni-plugins-njjqg\" (UID: \"afc9e28a-2764-4e91-884b-ead5986764cc\") " pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.612902 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.612855 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxlnk\" (UniqueName: \"kubernetes.io/projected/2be9a63e-5667-4342-a081-2554ff60db8a-kube-api-access-wxlnk\") pod \"aws-ebs-csi-driver-node-nbw76\" (UID: \"2be9a63e-5667-4342-a081-2554ff60db8a\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.616906 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.616865 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxc5z\" (UniqueName: \"kubernetes.io/projected/cea48c24-3755-41c7-b30c-5d60aa682341-kube-api-access-dxc5z\") pod \"multus-frmbq\" (UID: \"cea48c24-3755-41c7-b30c-5d60aa682341\") " pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.617753 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.617710 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9529\" (UniqueName: \"kubernetes.io/projected/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-kube-api-access-k9529\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:25.623352 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.623328 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwvfd\" (UniqueName: \"kubernetes.io/projected/427427dc-5248-4fdb-b49f-1309f98f2320-kube-api-access-rwvfd\") pod \"node-ca-qqpzf\" (UID: \"427427dc-5248-4fdb-b49f-1309f98f2320\") " pod="openshift-image-registry/node-ca-qqpzf" Jul 14 00:44:25.626849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.626828 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 00:44:25.678194 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.678143 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" event={"ID":"effd29edc9dfe1c426b92a89b21d0dbd","Type":"ContainerStarted","Data":"7273b65b9f467cbebbafdc42e5af0dc60a3dd73921e3eef2a956a9e37a8cdd87"} Jul 14 00:44:25.679565 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.679539 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-143.ec2.internal" event={"ID":"68f0c074f7c50b5b863d7294622d4fc6","Type":"ContainerStarted","Data":"0562b08e7a8b5d356ef451372c5d87122beac84ca39d3f522e13d196e6141ced"} Jul 14 00:44:25.700457 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700432 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srgfd\" (UniqueName: \"kubernetes.io/projected/d146a5e0-9ae6-4976-b3fb-26a5f033c39d-kube-api-access-srgfd\") pod \"iptables-alerter-hvmtb\" (UID: \"d146a5e0-9ae6-4976-b3fb-26a5f033c39d\") " pod="openshift-network-operator/iptables-alerter-hvmtb" Jul 14 00:44:25.700571 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700471 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-run-openvswitch\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.700571 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700496 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-node-log\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.700571 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700519 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2bae43db-11b2-47ff-951c-604be5df51cd-env-overrides\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.700571 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700557 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-systemd\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.700782 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700614 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-node-log\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.700782 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700629 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-run-openvswitch\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.700782 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700674 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-run\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.700782 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700701 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-lib-modules\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.700782 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700730 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-var-lib-kubelet\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.700782 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700755 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-tuned\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.701088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700784 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d146a5e0-9ae6-4976-b3fb-26a5f033c39d-iptables-alerter-script\") pod \"iptables-alerter-hvmtb\" (UID: \"d146a5e0-9ae6-4976-b3fb-26a5f033c39d\") " pod="openshift-network-operator/iptables-alerter-hvmtb" Jul 14 00:44:25.701088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700809 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-kubernetes\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.701088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700839 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-run-netns\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.701088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700859 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-run\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.701088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700984 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-lib-modules\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.701088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701053 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-var-lib-kubelet\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.701088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700864 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-sysctl-conf\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701097 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-var-lib-openvswitch\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701122 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b65z2\" (UniqueName: \"kubernetes.io/projected/2bae43db-11b2-47ff-951c-604be5df51cd-kube-api-access-b65z2\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701146 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-sysctl-d\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701172 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d146a5e0-9ae6-4976-b3fb-26a5f033c39d-host-slash\") pod \"iptables-alerter-hvmtb\" (UID: \"d146a5e0-9ae6-4976-b3fb-26a5f033c39d\") " pod="openshift-network-operator/iptables-alerter-hvmtb" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701202 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2bae43db-11b2-47ff-951c-604be5df51cd-env-overrides\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.700794 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-systemd\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701215 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2bae43db-11b2-47ff-951c-604be5df51cd-ovnkube-config\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701286 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-cni-netd\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701284 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-kubernetes\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701317 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701346 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-sys\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701296 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-run-netns\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701374 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnlj4\" (UniqueName: \"kubernetes.io/projected/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-kube-api-access-nnlj4\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701393 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-cni-netd\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.701447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701438 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-etc-openvswitch\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701466 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701488 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d146a5e0-9ae6-4976-b3fb-26a5f033c39d-iptables-alerter-script\") pod \"iptables-alerter-hvmtb\" (UID: \"d146a5e0-9ae6-4976-b3fb-26a5f033c39d\") " pod="openshift-network-operator/iptables-alerter-hvmtb" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701490 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-systemd-units\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701539 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701550 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-kubelet\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701553 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-systemd-units\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701594 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-sys\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701345 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d146a5e0-9ae6-4976-b3fb-26a5f033c39d-host-slash\") pod \"iptables-alerter-hvmtb\" (UID: \"d146a5e0-9ae6-4976-b3fb-26a5f033c39d\") " pod="openshift-network-operator/iptables-alerter-hvmtb" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701603 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-run-ovn\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701639 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-run-ovn\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701637 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-slash\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701668 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-slash\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701679 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-cni-bin\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701691 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-sysctl-conf\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701717 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-sysconfig\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701751 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-run-systemd\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701782 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-log-socket\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.702172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701810 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-sysctl-d\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701812 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2bae43db-11b2-47ff-951c-604be5df51cd-ovnkube-script-lib\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701838 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2bae43db-11b2-47ff-951c-604be5df51cd-ovnkube-config\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701207 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-var-lib-openvswitch\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701872 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2bae43db-11b2-47ff-951c-604be5df51cd-ovn-node-metrics-cert\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701898 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-etc-openvswitch\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701902 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmjsd\" (UniqueName: \"kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd\") pod \"network-check-target-tzdlx\" (UID: \"38610aec-8e4e-492e-8f63-1e34067176d0\") " pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701910 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-run-ovn-kubernetes\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701928 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-modprobe-d\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701953 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-host\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701939 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-cni-bin\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701980 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-tmp\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701998 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-sysconfig\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.702061 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-host-kubelet\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.702113 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-modprobe-d\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.701956 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-run-systemd\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.702320 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-host\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.702333 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2bae43db-11b2-47ff-951c-604be5df51cd-ovnkube-script-lib\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.703122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.702362 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2bae43db-11b2-47ff-951c-604be5df51cd-log-socket\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.706029 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.706004 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-etc-tuned\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.706306 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.706281 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2bae43db-11b2-47ff-951c-604be5df51cd-ovn-node-metrics-cert\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.707079 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:25.707007 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:44:25.707079 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:25.707030 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:44:25.707079 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:25.707045 2557 projected.go:196] Error preparing data for projected volume kube-api-access-gmjsd for pod openshift-network-diagnostics/network-check-target-tzdlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:25.707294 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:25.707122 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd podName:38610aec-8e4e-492e-8f63-1e34067176d0 nodeName:}" failed. No retries permitted until 2026-07-14 00:44:26.207103572 +0000 UTC m=+3.058470943 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gmjsd" (UniqueName: "kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd") pod "network-check-target-tzdlx" (UID: "38610aec-8e4e-492e-8f63-1e34067176d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:25.707953 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.707724 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-tmp\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.710348 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.710320 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnlj4\" (UniqueName: \"kubernetes.io/projected/9d1a87f1-7ec0-4355-a93d-e1985c1a77d2-kube-api-access-nnlj4\") pod \"tuned-l9mwk\" (UID: \"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2\") " pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.712778 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.710545 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srgfd\" (UniqueName: \"kubernetes.io/projected/d146a5e0-9ae6-4976-b3fb-26a5f033c39d-kube-api-access-srgfd\") pod \"iptables-alerter-hvmtb\" (UID: \"d146a5e0-9ae6-4976-b3fb-26a5f033c39d\") " pod="openshift-network-operator/iptables-alerter-hvmtb" Jul 14 00:44:25.712778 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.710673 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b65z2\" (UniqueName: \"kubernetes.io/projected/2bae43db-11b2-47ff-951c-604be5df51cd-kube-api-access-b65z2\") pod \"ovnkube-node-v9t68\" (UID: \"2bae43db-11b2-47ff-951c-604be5df51cd\") " pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.794386 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.794354 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" Jul 14 00:44:25.803710 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.803690 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-frmbq" Jul 14 00:44:25.804122 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:25.804095 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be9a63e_5667_4342_a081_2554ff60db8a.slice/crio-7b2655bad1b7a6a1a2704eb9be555ea3177c740d73dde763289cad682c462da0 WatchSource:0}: Error finding container 7b2655bad1b7a6a1a2704eb9be555ea3177c740d73dde763289cad682c462da0: Status 404 returned error can't find the container with id 7b2655bad1b7a6a1a2704eb9be555ea3177c740d73dde763289cad682c462da0 Jul 14 00:44:25.812293 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:25.812268 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea48c24_3755_41c7_b30c_5d60aa682341.slice/crio-b0e8770adbae7251123cbeddeb1f68fea7e3383ff4b1c09f8cc41d623e65aa88 WatchSource:0}: Error finding container b0e8770adbae7251123cbeddeb1f68fea7e3383ff4b1c09f8cc41d623e65aa88: Status 404 returned error can't find the container with id b0e8770adbae7251123cbeddeb1f68fea7e3383ff4b1c09f8cc41d623e65aa88 Jul 14 00:44:25.813247 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.813230 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7xct5" Jul 14 00:44:25.820598 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.820579 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-njjqg" Jul 14 00:44:25.823155 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:25.823131 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8dacd2e_6336_4956_aa9f_e9a0964534d0.slice/crio-426800f09cbc9f34171fd88cf94c3b0da0c22ee44c69cf611334f603b9aac3cd WatchSource:0}: Error finding container 426800f09cbc9f34171fd88cf94c3b0da0c22ee44c69cf611334f603b9aac3cd: Status 404 returned error can't find the container with id 426800f09cbc9f34171fd88cf94c3b0da0c22ee44c69cf611334f603b9aac3cd Jul 14 00:44:25.829340 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.829323 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:25.830257 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:25.830233 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc9e28a_2764_4e91_884b_ead5986764cc.slice/crio-fa958221a1f091a863c4567f123870cf875ab75dbd884701a16fea042bb46d76 WatchSource:0}: Error finding container fa958221a1f091a863c4567f123870cf875ab75dbd884701a16fea042bb46d76: Status 404 returned error can't find the container with id fa958221a1f091a863c4567f123870cf875ab75dbd884701a16fea042bb46d76 Jul 14 00:44:25.836637 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.836604 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qqpzf" Jul 14 00:44:25.840971 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:25.840944 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4febc6f_5f93_4d99_bf2c_1a4211db67f8.slice/crio-51811782d03c11c01745b715fbc8573811566d64ba2b98bf7323c3b55000c0b3 WatchSource:0}: Error finding container 51811782d03c11c01745b715fbc8573811566d64ba2b98bf7323c3b55000c0b3: Status 404 returned error can't find the container with id 51811782d03c11c01745b715fbc8573811566d64ba2b98bf7323c3b55000c0b3 Jul 14 00:44:25.845873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.845853 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hvmtb" Jul 14 00:44:25.846014 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:25.845990 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod427427dc_5248_4fdb_b49f_1309f98f2320.slice/crio-610fb2a166fed6b929f0a13967bd9f2521bcde27652cb834e088ee262e1597af WatchSource:0}: Error finding container 610fb2a166fed6b929f0a13967bd9f2521bcde27652cb834e088ee262e1597af: Status 404 returned error can't find the container with id 610fb2a166fed6b929f0a13967bd9f2521bcde27652cb834e088ee262e1597af Jul 14 00:44:25.855807 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:25.855780 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd146a5e0_9ae6_4976_b3fb_26a5f033c39d.slice/crio-66faf94d82f6fe05d0aede19cc0f4867fd82dd18418cd13222c31f79c52a59b7 WatchSource:0}: Error finding container 66faf94d82f6fe05d0aede19cc0f4867fd82dd18418cd13222c31f79c52a59b7: Status 404 returned error can't find the container with id 66faf94d82f6fe05d0aede19cc0f4867fd82dd18418cd13222c31f79c52a59b7 Jul 14 00:44:25.856904 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.856884 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:25.862988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:25.862966 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" Jul 14 00:44:25.867079 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:25.867054 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bae43db_11b2_47ff_951c_604be5df51cd.slice/crio-62dbda2fcad468ec2b387bf7a2c3e6ba1393e9bb1d4e061022d0acf5a34e18dd WatchSource:0}: Error finding container 62dbda2fcad468ec2b387bf7a2c3e6ba1393e9bb1d4e061022d0acf5a34e18dd: Status 404 returned error can't find the container with id 62dbda2fcad468ec2b387bf7a2c3e6ba1393e9bb1d4e061022d0acf5a34e18dd Jul 14 00:44:25.874644 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:25.874607 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1a87f1_7ec0_4355_a93d_e1985c1a77d2.slice/crio-344386e758e900a809d39c072d7dbd822a28cc07a930f66c0b97d3193d218abb WatchSource:0}: Error finding container 344386e758e900a809d39c072d7dbd822a28cc07a930f66c0b97d3193d218abb: Status 404 returned error can't find the container with id 344386e758e900a809d39c072d7dbd822a28cc07a930f66c0b97d3193d218abb Jul 14 00:44:26.105085 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.104785 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:26.105085 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.105069 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:26.105341 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:26.105000 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:26.105341 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:26.105194 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret podName:5d75eb52-5424-4c0e-a6bc-1fe4d642587a nodeName:}" failed. No retries permitted until 2026-07-14 00:44:27.105168285 +0000 UTC m=+3.956535655 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret") pod "global-pull-secret-syncer-nzt74" (UID: "5d75eb52-5424-4c0e-a6bc-1fe4d642587a") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:26.105341 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:26.105214 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:26.105341 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:26.105276 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs podName:4f92c479-1507-4bd3-a7a3-0bcdb89a049f nodeName:}" failed. No retries permitted until 2026-07-14 00:44:27.105258538 +0000 UTC m=+3.956625890 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs") pod "network-metrics-daemon-txzdm" (UID: "4f92c479-1507-4bd3-a7a3-0bcdb89a049f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:26.306319 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.306284 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmjsd\" (UniqueName: \"kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd\") pod \"network-check-target-tzdlx\" (UID: \"38610aec-8e4e-492e-8f63-1e34067176d0\") " pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:26.306506 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:26.306457 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:44:26.306506 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:26.306481 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:44:26.306506 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:26.306492 2557 projected.go:196] Error preparing data for projected volume kube-api-access-gmjsd for pod openshift-network-diagnostics/network-check-target-tzdlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:26.306681 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:26.306556 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd podName:38610aec-8e4e-492e-8f63-1e34067176d0 nodeName:}" failed. No retries permitted until 2026-07-14 00:44:27.306541858 +0000 UTC m=+4.157909211 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gmjsd" (UniqueName: "kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd") pod "network-check-target-tzdlx" (UID: "38610aec-8e4e-492e-8f63-1e34067176d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:26.534955 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.534882 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 00:39:25 +0000 UTC" deadline="2028-02-10 16:46:48.050406749 +0000 UTC" Jul 14 00:44:26.534955 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.534937 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13840h2m21.515474328s" Jul 14 00:44:26.672284 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.672247 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:26.672460 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:26.672395 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:26.682762 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.682697 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" event={"ID":"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2","Type":"ContainerStarted","Data":"344386e758e900a809d39c072d7dbd822a28cc07a930f66c0b97d3193d218abb"} Jul 14 00:44:26.684086 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.684050 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9dwkf" event={"ID":"b4febc6f-5f93-4d99-bf2c-1a4211db67f8","Type":"ContainerStarted","Data":"51811782d03c11c01745b715fbc8573811566d64ba2b98bf7323c3b55000c0b3"} Jul 14 00:44:26.685712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.685686 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njjqg" event={"ID":"afc9e28a-2764-4e91-884b-ead5986764cc","Type":"ContainerStarted","Data":"fa958221a1f091a863c4567f123870cf875ab75dbd884701a16fea042bb46d76"} Jul 14 00:44:26.687222 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.687157 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-frmbq" event={"ID":"cea48c24-3755-41c7-b30c-5d60aa682341","Type":"ContainerStarted","Data":"b0e8770adbae7251123cbeddeb1f68fea7e3383ff4b1c09f8cc41d623e65aa88"} Jul 14 00:44:26.688371 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.688346 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" event={"ID":"2be9a63e-5667-4342-a081-2554ff60db8a","Type":"ContainerStarted","Data":"7b2655bad1b7a6a1a2704eb9be555ea3177c740d73dde763289cad682c462da0"} Jul 14 00:44:26.689952 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.689928 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" event={"ID":"2bae43db-11b2-47ff-951c-604be5df51cd","Type":"ContainerStarted","Data":"62dbda2fcad468ec2b387bf7a2c3e6ba1393e9bb1d4e061022d0acf5a34e18dd"} Jul 14 00:44:26.691328 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.691298 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hvmtb" event={"ID":"d146a5e0-9ae6-4976-b3fb-26a5f033c39d","Type":"ContainerStarted","Data":"66faf94d82f6fe05d0aede19cc0f4867fd82dd18418cd13222c31f79c52a59b7"} Jul 14 00:44:26.692788 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.692760 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qqpzf" event={"ID":"427427dc-5248-4fdb-b49f-1309f98f2320","Type":"ContainerStarted","Data":"610fb2a166fed6b929f0a13967bd9f2521bcde27652cb834e088ee262e1597af"} Jul 14 00:44:26.694254 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:26.694225 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7xct5" event={"ID":"c8dacd2e-6336-4956-aa9f-e9a0964534d0","Type":"ContainerStarted","Data":"426800f09cbc9f34171fd88cf94c3b0da0c22ee44c69cf611334f603b9aac3cd"} Jul 14 00:44:27.114648 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:27.113843 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:27.114648 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:27.113937 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:27.114648 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:27.114085 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:27.114648 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:27.114150 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret podName:5d75eb52-5424-4c0e-a6bc-1fe4d642587a nodeName:}" failed. No retries permitted until 2026-07-14 00:44:29.114130735 +0000 UTC m=+5.965498140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret") pod "global-pull-secret-syncer-nzt74" (UID: "5d75eb52-5424-4c0e-a6bc-1fe4d642587a") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:27.114648 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:27.114245 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:27.114648 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:27.114284 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs podName:4f92c479-1507-4bd3-a7a3-0bcdb89a049f nodeName:}" failed. No retries permitted until 2026-07-14 00:44:29.114271366 +0000 UTC m=+5.965638719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs") pod "network-metrics-daemon-txzdm" (UID: "4f92c479-1507-4bd3-a7a3-0bcdb89a049f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:27.315526 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:27.315488 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmjsd\" (UniqueName: \"kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd\") pod \"network-check-target-tzdlx\" (UID: \"38610aec-8e4e-492e-8f63-1e34067176d0\") " pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:27.315717 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:27.315662 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:44:27.315717 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:27.315686 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:44:27.315717 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:27.315701 2557 projected.go:196] Error preparing data for projected volume kube-api-access-gmjsd for pod openshift-network-diagnostics/network-check-target-tzdlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:27.315949 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:27.315762 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd podName:38610aec-8e4e-492e-8f63-1e34067176d0 nodeName:}" failed. No retries permitted until 2026-07-14 00:44:29.315741044 +0000 UTC m=+6.167108421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gmjsd" (UniqueName: "kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd") pod "network-check-target-tzdlx" (UID: "38610aec-8e4e-492e-8f63-1e34067176d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:27.538349 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:27.535774 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 00:39:25 +0000 UTC" deadline="2028-01-01 20:30:55.147525458 +0000 UTC" Jul 14 00:44:27.538349 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:27.535812 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12883h46m27.611718315s" Jul 14 00:44:27.672368 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:27.672289 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:27.672537 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:27.672414 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:27.675210 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:27.675183 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:27.675339 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:27.675321 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:27.705751 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:27.705718 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-143.ec2.internal" event={"ID":"68f0c074f7c50b5b863d7294622d4fc6","Type":"ContainerStarted","Data":"5524efcbc48f77b0a5619a47b778784e007724ce3eda3e5e4c64c09cf8c2b54a"} Jul 14 00:44:27.717790 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:27.717732 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-143.ec2.internal" podStartSLOduration=2.717712073 podStartE2EDuration="2.717712073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:44:27.7169275 +0000 UTC m=+4.568294876" watchObservedRunningTime="2026-07-14 00:44:27.717712073 +0000 UTC m=+4.569079449" Jul 14 00:44:28.672274 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:28.672239 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:28.672757 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:28.672395 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:28.723328 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:28.723292 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" event={"ID":"effd29edc9dfe1c426b92a89b21d0dbd","Type":"ContainerStarted","Data":"ca0b77d3e94eab72c5bd6f605c4ab1b5309bd6d8f2419c161b72f8f32bf087ae"} Jul 14 00:44:29.131631 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:29.131571 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:29.131814 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:29.131662 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:29.131899 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:29.131827 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:29.131899 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:29.131893 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret podName:5d75eb52-5424-4c0e-a6bc-1fe4d642587a nodeName:}" failed. No retries permitted until 2026-07-14 00:44:33.131874307 +0000 UTC m=+9.983241681 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret") pod "global-pull-secret-syncer-nzt74" (UID: "5d75eb52-5424-4c0e-a6bc-1fe4d642587a") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:29.132352 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:29.132329 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:29.132468 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:29.132394 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs podName:4f92c479-1507-4bd3-a7a3-0bcdb89a049f nodeName:}" failed. No retries permitted until 2026-07-14 00:44:33.132374737 +0000 UTC m=+9.983742093 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs") pod "network-metrics-daemon-txzdm" (UID: "4f92c479-1507-4bd3-a7a3-0bcdb89a049f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:29.334328 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:29.333609 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmjsd\" (UniqueName: \"kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd\") pod \"network-check-target-tzdlx\" (UID: \"38610aec-8e4e-492e-8f63-1e34067176d0\") " pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:29.334328 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:29.333851 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:44:29.334328 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:29.333876 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:44:29.334328 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:29.333890 2557 projected.go:196] Error preparing data for projected volume kube-api-access-gmjsd for pod openshift-network-diagnostics/network-check-target-tzdlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:29.334328 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:29.333947 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd podName:38610aec-8e4e-492e-8f63-1e34067176d0 nodeName:}" failed. No retries permitted until 2026-07-14 00:44:33.333928288 +0000 UTC m=+10.185295652 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gmjsd" (UniqueName: "kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd") pod "network-check-target-tzdlx" (UID: "38610aec-8e4e-492e-8f63-1e34067176d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:29.674846 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:29.674146 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:29.674846 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:29.674293 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:29.674846 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:29.674705 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:29.674846 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:29.674801 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:30.672676 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:30.672179 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:30.672676 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:30.672318 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:31.671977 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:31.671911 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:31.672413 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:31.671917 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:31.672413 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:31.672129 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:31.672413 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:31.672244 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:31.730775 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:31.730745 2557 generic.go:356] "Generic (PLEG): container finished" podID="effd29edc9dfe1c426b92a89b21d0dbd" containerID="ca0b77d3e94eab72c5bd6f605c4ab1b5309bd6d8f2419c161b72f8f32bf087ae" exitCode=0 Jul 14 00:44:31.730948 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:31.730798 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" event={"ID":"effd29edc9dfe1c426b92a89b21d0dbd","Type":"ContainerDied","Data":"ca0b77d3e94eab72c5bd6f605c4ab1b5309bd6d8f2419c161b72f8f32bf087ae"} Jul 14 00:44:32.672043 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:32.671907 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:32.672525 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:32.672046 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:33.164165 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:33.164123 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:33.164340 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:33.164200 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:33.164340 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:33.164315 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:33.164448 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:33.164382 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret podName:5d75eb52-5424-4c0e-a6bc-1fe4d642587a nodeName:}" failed. No retries permitted until 2026-07-14 00:44:41.16436418 +0000 UTC m=+18.015731537 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret") pod "global-pull-secret-syncer-nzt74" (UID: "5d75eb52-5424-4c0e-a6bc-1fe4d642587a") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:33.164448 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:33.164315 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:33.164448 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:33.164436 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs podName:4f92c479-1507-4bd3-a7a3-0bcdb89a049f nodeName:}" failed. No retries permitted until 2026-07-14 00:44:41.164423857 +0000 UTC m=+18.015791208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs") pod "network-metrics-daemon-txzdm" (UID: "4f92c479-1507-4bd3-a7a3-0bcdb89a049f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:33.365322 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:33.365286 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmjsd\" (UniqueName: \"kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd\") pod \"network-check-target-tzdlx\" (UID: \"38610aec-8e4e-492e-8f63-1e34067176d0\") " pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:33.365486 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:33.365448 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:44:33.365554 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:33.365496 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:44:33.365554 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:33.365509 2557 projected.go:196] Error preparing data for projected volume kube-api-access-gmjsd for pod openshift-network-diagnostics/network-check-target-tzdlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:33.365642 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:33.365567 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd podName:38610aec-8e4e-492e-8f63-1e34067176d0 nodeName:}" failed. No retries permitted until 2026-07-14 00:44:41.365547247 +0000 UTC m=+18.216914614 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gmjsd" (UniqueName: "kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd") pod "network-check-target-tzdlx" (UID: "38610aec-8e4e-492e-8f63-1e34067176d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:33.675656 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:33.675588 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:33.676095 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:33.675740 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:33.676535 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:33.676259 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:33.676535 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:33.676413 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:34.672458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:34.672430 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:34.672623 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:34.672544 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:35.672084 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:35.671628 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:35.672084 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:35.671750 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:35.672614 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:35.672301 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:35.672614 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:35.672407 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:36.672041 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:36.672012 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:36.672195 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:36.672107 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:37.672519 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:37.672491 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:37.672949 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:37.672494 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:37.672949 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:37.672613 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:37.672949 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:37.672668 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:38.672195 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:38.672144 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:38.672356 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:38.672281 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:39.672511 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:39.672485 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:39.672511 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:39.672518 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:39.672970 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:39.672585 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:39.672970 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:39.672696 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:40.671603 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:40.671578 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:40.671800 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:40.671716 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:41.230297 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:41.230268 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:41.230609 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:41.230314 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:41.230609 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:41.230419 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:41.230609 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:41.230469 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret podName:5d75eb52-5424-4c0e-a6bc-1fe4d642587a nodeName:}" failed. No retries permitted until 2026-07-14 00:44:57.23045517 +0000 UTC m=+34.081822523 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret") pod "global-pull-secret-syncer-nzt74" (UID: "5d75eb52-5424-4c0e-a6bc-1fe4d642587a") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:41.230609 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:41.230419 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:41.230609 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:41.230538 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs podName:4f92c479-1507-4bd3-a7a3-0bcdb89a049f nodeName:}" failed. No retries permitted until 2026-07-14 00:44:57.230526067 +0000 UTC m=+34.081893414 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs") pod "network-metrics-daemon-txzdm" (UID: "4f92c479-1507-4bd3-a7a3-0bcdb89a049f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:41.431620 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:41.431539 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmjsd\" (UniqueName: \"kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd\") pod \"network-check-target-tzdlx\" (UID: \"38610aec-8e4e-492e-8f63-1e34067176d0\") " pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:41.431767 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:41.431718 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:44:41.431767 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:41.431745 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:44:41.431767 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:41.431760 2557 projected.go:196] Error preparing data for projected volume kube-api-access-gmjsd for pod openshift-network-diagnostics/network-check-target-tzdlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:41.431896 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:41.431821 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd podName:38610aec-8e4e-492e-8f63-1e34067176d0 nodeName:}" failed. No retries permitted until 2026-07-14 00:44:57.431801794 +0000 UTC m=+34.283169146 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gmjsd" (UniqueName: "kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd") pod "network-check-target-tzdlx" (UID: "38610aec-8e4e-492e-8f63-1e34067176d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:41.672988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:41.672500 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:41.672988 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:41.672628 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:41.672988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:41.672691 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:41.672988 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:41.672808 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:42.671786 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:42.671755 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:42.672183 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:42.671894 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:43.672504 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:43.672475 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:43.672919 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:43.672569 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:43.672919 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:43.672654 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:43.672919 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:43.672737 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:44.672628 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:44.672604 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:44.673054 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:44.672730 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:45.672559 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.672388 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:45.672698 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.672439 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:45.672698 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:45.672670 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:45.673358 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:45.672722 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:45.757314 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.757142 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" event={"ID":"effd29edc9dfe1c426b92a89b21d0dbd","Type":"ContainerStarted","Data":"a7c40f3d79698bf6852ea85988551027995da0e8a9164523e9046603994bd2b3"} Jul 14 00:44:45.760814 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.760796 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 00:44:45.761278 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.761254 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" event={"ID":"2bae43db-11b2-47ff-951c-604be5df51cd","Type":"ContainerStarted","Data":"a071badbaeff14ac25206d108793bb7841a6b6967e7afa5d0261bb0663cec56e"} Jul 14 00:44:45.761385 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.761325 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" event={"ID":"2bae43db-11b2-47ff-951c-604be5df51cd","Type":"ContainerStarted","Data":"5713f94ebb8ee140f4f12a61194b6a1c09bcf8cef4520a9527174e25ce83093d"} Jul 14 00:44:45.762691 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.762673 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qqpzf" event={"ID":"427427dc-5248-4fdb-b49f-1309f98f2320","Type":"ContainerStarted","Data":"c19615cf671b4751c0901945e84fa52f10d61d5432aedbbf0a2670d419842765"} Jul 14 00:44:45.763969 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.763947 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7xct5" event={"ID":"c8dacd2e-6336-4956-aa9f-e9a0964534d0","Type":"ContainerStarted","Data":"19e85cae8225b178ca42d5763c5e056f125a876bb5a0413463caefa14b08c5db"} Jul 14 00:44:45.765383 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.765364 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" event={"ID":"9d1a87f1-7ec0-4355-a93d-e1985c1a77d2","Type":"ContainerStarted","Data":"c316c92563e3cd07f5b4d0552d1814b13b8a4494ee5b526514147f3dfbdb3dd1"} Jul 14 00:44:45.766789 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.766767 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-9dwkf" event={"ID":"b4febc6f-5f93-4d99-bf2c-1a4211db67f8","Type":"ContainerStarted","Data":"5c859cc2d620a66b377de16b4b1ab2d5c44ef05770bafcc2e6656c9886b60ee9"} Jul 14 00:44:45.768266 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.768245 2557 generic.go:356] "Generic (PLEG): container finished" podID="afc9e28a-2764-4e91-884b-ead5986764cc" containerID="9a800a7b7c9d6cdd1a4cdbe06d67f44abc44ff409e5aaf9f23bb08e76e4b9bd2" exitCode=0 Jul 14 00:44:45.768359 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.768314 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njjqg" event={"ID":"afc9e28a-2764-4e91-884b-ead5986764cc","Type":"ContainerDied","Data":"9a800a7b7c9d6cdd1a4cdbe06d67f44abc44ff409e5aaf9f23bb08e76e4b9bd2"} Jul 14 00:44:45.769914 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.769881 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-frmbq" event={"ID":"cea48c24-3755-41c7-b30c-5d60aa682341","Type":"ContainerStarted","Data":"36cfffd28e98ea9816470bf2cad9d2d6fbc118f95a0c8f2ed98850490c741a04"} Jul 14 00:44:45.771325 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.771305 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" event={"ID":"2be9a63e-5667-4342-a081-2554ff60db8a","Type":"ContainerStarted","Data":"9d261021ab46998412dcdbb1ca0532b72213b228edb57715d544bc7f33e39519"} Jul 14 00:44:45.779542 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.779506 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-143.ec2.internal" podStartSLOduration=20.779495464 podStartE2EDuration="20.779495464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:44:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:44:45.779309626 +0000 UTC m=+22.630676997" watchObservedRunningTime="2026-07-14 00:44:45.779495464 +0000 UTC m=+22.630862835" Jul 14 00:44:45.809098 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.809063 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-9dwkf" podStartSLOduration=2.409285592 podStartE2EDuration="21.809049084s" totalImagesPullingTime="19.399763492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:44:45.808906763 +0000 UTC m=+22.660274134" watchObservedRunningTime="2026-07-14 00:44:45.809049084 +0000 UTC m=+22.660416453" Jul 14 00:44:45.819088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.819058 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qqpzf" podStartSLOduration=10.811929706 podStartE2EDuration="21.819047223s" totalImagesPullingTime="11.007117517s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:44:45.818904967 +0000 UTC m=+22.670272338" watchObservedRunningTime="2026-07-14 00:44:45.819047223 +0000 UTC m=+22.670414594" Jul 14 00:44:45.834458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.834418 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-frmbq" podStartSLOduration=2.349909324 podStartE2EDuration="21.834405426s" totalImagesPullingTime="19.484496102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:44:45.833233604 +0000 UTC m=+22.684600974" watchObservedRunningTime="2026-07-14 00:44:45.834405426 +0000 UTC m=+22.685772796" Jul 14 00:44:45.858807 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.858740 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l9mwk" podStartSLOduration=2.471816776 podStartE2EDuration="21.858728404s" totalImagesPullingTime="19.386911628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:44:45.846746918 +0000 UTC m=+22.698114289" watchObservedRunningTime="2026-07-14 00:44:45.858728404 +0000 UTC m=+22.710095774" Jul 14 00:44:45.858977 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.858960 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7xct5" podStartSLOduration=2.42431933 podStartE2EDuration="21.858954894s" totalImagesPullingTime="19.434635564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:44:45.858657439 +0000 UTC m=+22.710024808" watchObservedRunningTime="2026-07-14 00:44:45.858954894 +0000 UTC m=+22.710322264" Jul 14 00:44:45.904709 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.904685 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:45.982509 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:45.982380 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:46.084756 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.084732 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:46.112617 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.112562 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:46.113097 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.113081 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:46.672168 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.672122 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:46.672336 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:46.672270 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:46.775099 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.775034 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hvmtb" event={"ID":"d146a5e0-9ae6-4976-b3fb-26a5f033c39d","Type":"ContainerStarted","Data":"49bd9dcc042637ee1e49a75307f294eadb6ec943531194765803591be1966544"} Jul 14 00:44:46.778099 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.778079 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 00:44:46.778474 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.778453 2557 generic.go:356] "Generic (PLEG): container finished" podID="2bae43db-11b2-47ff-951c-604be5df51cd" containerID="a071badbaeff14ac25206d108793bb7841a6b6967e7afa5d0261bb0663cec56e" exitCode=1 Jul 14 00:44:46.778587 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.778560 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" event={"ID":"2bae43db-11b2-47ff-951c-604be5df51cd","Type":"ContainerDied","Data":"a071badbaeff14ac25206d108793bb7841a6b6967e7afa5d0261bb0663cec56e"} Jul 14 00:44:46.778661 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.778640 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" event={"ID":"2bae43db-11b2-47ff-951c-604be5df51cd","Type":"ContainerStarted","Data":"9231af671366eae25697d6cda193b694a866ae3bdfaa5292d009740002fd5ac7"} Jul 14 00:44:46.778722 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.778690 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" event={"ID":"2bae43db-11b2-47ff-951c-604be5df51cd","Type":"ContainerStarted","Data":"8c955afc19cc5b55ada0709c79e9122003f243e46f691566794aacee166e0225"} Jul 14 00:44:46.779217 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.779196 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:46.779295 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.779232 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" event={"ID":"2bae43db-11b2-47ff-951c-604be5df51cd","Type":"ContainerStarted","Data":"4439fdede4414b209ade74def97dcb23bb3724078e08fe61efc356fb504d9482"} Jul 14 00:44:46.779295 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.779287 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" event={"ID":"2bae43db-11b2-47ff-951c-604be5df51cd","Type":"ContainerStarted","Data":"ee8bbbd684f64700eb6e4ab6ddc56279cf429a8824cf1c32fd0ce5ac121ed4b9"} Jul 14 00:44:46.779379 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.779345 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-9dwkf" Jul 14 00:44:46.799708 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.799650 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hvmtb" podStartSLOduration=3.395984974 podStartE2EDuration="22.799632114s" totalImagesPullingTime="19.40364714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:44:46.7878199 +0000 UTC m=+23.639187275" watchObservedRunningTime="2026-07-14 00:44:46.799632114 +0000 UTC m=+23.650999487" Jul 14 00:44:46.904989 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.904786 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:46.959559 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.959526 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:46.990768 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:46.990744 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 00:44:47.558040 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:47.557921 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T00:44:46.990765498Z","UUID":"901f5d29-c436-4031-a52d-9bbdc933ff17","Handler":null,"Name":"","Endpoint":""} Jul 14 00:44:47.561278 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:47.561257 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 00:44:47.561406 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:47.561293 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 00:44:47.672521 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:47.672483 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:47.672675 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:47.672595 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:47.672769 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:47.672750 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:47.672873 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:47.672853 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:47.783499 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:47.783467 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" event={"ID":"2be9a63e-5667-4342-a081-2554ff60db8a","Type":"ContainerStarted","Data":"df5337c9db47ab919697f0c3a22e84903adaf5422528a47dab421de016e7ed4f"} Jul 14 00:44:48.672537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:48.672463 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:48.672691 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:48.672569 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:48.788080 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:48.788037 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" event={"ID":"2be9a63e-5667-4342-a081-2554ff60db8a","Type":"ContainerStarted","Data":"2b974269aa7ff9b3e74dc3abff71eabee09d545089eabe6165c49df28a2e0af0"} Jul 14 00:44:48.791494 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:48.791469 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 00:44:48.791860 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:48.791833 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" event={"ID":"2bae43db-11b2-47ff-951c-604be5df51cd","Type":"ContainerStarted","Data":"8ee217efb83d308744dee3b7f5bde3cb296b3e642365d0258962632150cacc26"} Jul 14 00:44:48.803185 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:48.803126 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nbw76" podStartSLOduration=7.060031335 podStartE2EDuration="24.803109604s" totalImagesPullingTime="17.743078269s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:44:48.803032226 +0000 UTC m=+25.654399598" watchObservedRunningTime="2026-07-14 00:44:48.803109604 +0000 UTC m=+25.654476956" Jul 14 00:44:49.671767 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:49.671732 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:49.671934 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:49.671861 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:49.671999 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:49.671946 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:49.672059 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:49.672036 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:50.014254 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.014181 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:50.116352 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.116332 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:50.216702 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.216672 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:50.274349 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.274291 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:50.672732 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.672606 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:50.672863 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:50.672840 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:50.797878 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.797849 2557 generic.go:356] "Generic (PLEG): container finished" podID="afc9e28a-2764-4e91-884b-ead5986764cc" containerID="21db34c6ed18d63003a9e0c29a315314c045eef7b3457de0f4ede4bb68ab4a0b" exitCode=0 Jul 14 00:44:50.797984 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.797939 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njjqg" event={"ID":"afc9e28a-2764-4e91-884b-ead5986764cc","Type":"ContainerDied","Data":"21db34c6ed18d63003a9e0c29a315314c045eef7b3457de0f4ede4bb68ab4a0b"} Jul 14 00:44:50.801646 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.801629 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 00:44:50.802049 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.802028 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" event={"ID":"2bae43db-11b2-47ff-951c-604be5df51cd","Type":"ContainerStarted","Data":"990a96c1951f555c2f1d651a5b8f72900be6c771e7f6dc1dbd0fb5c17ec7e6df"} Jul 14 00:44:50.802366 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.802341 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:50.802418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.802402 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:50.802538 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.802508 2557 scope.go:117] "RemoveContainer" containerID="a071badbaeff14ac25206d108793bb7841a6b6967e7afa5d0261bb0663cec56e" Jul 14 00:44:50.818847 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.818830 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:50.959679 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:50.959650 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:51.011627 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:51.011599 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:51.672354 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:51.672323 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:51.672840 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:51.672441 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:51.672840 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:51.672502 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:51.672840 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:51.672605 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:51.808588 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:51.808563 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 00:44:51.808957 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:51.808937 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" event={"ID":"2bae43db-11b2-47ff-951c-604be5df51cd","Type":"ContainerStarted","Data":"45ab4e6ed21e34cef4335931bb1844f1cd10cb5eb57277f159fa3d7ae0481362"} Jul 14 00:44:51.809137 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:51.809106 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:51.823284 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:51.823261 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:44:51.832747 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:51.832713 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" podStartSLOduration=8.219602268 podStartE2EDuration="27.83270081s" totalImagesPullingTime="19.613098542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:44:51.832245854 +0000 UTC m=+28.683613223" watchObservedRunningTime="2026-07-14 00:44:51.83270081 +0000 UTC m=+28.684068180" Jul 14 00:44:52.586594 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:52.586429 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tzdlx"] Jul 14 00:44:52.586742 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:52.586661 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:52.586783 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:52.586752 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:52.589623 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:52.589596 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nzt74"] Jul 14 00:44:52.589728 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:52.589685 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:52.589802 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:52.589782 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:52.592444 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:52.590584 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-txzdm"] Jul 14 00:44:52.592444 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:52.590718 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:52.592444 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:52.590927 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:52.812746 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:52.812724 2557 generic.go:356] "Generic (PLEG): container finished" podID="afc9e28a-2764-4e91-884b-ead5986764cc" containerID="3fb4bfa15e0d3e345d0cd2f6f5f7f8b03793fa461ba39b105bf1431998d5306b" exitCode=0 Jul 14 00:44:52.813275 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:52.812817 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njjqg" event={"ID":"afc9e28a-2764-4e91-884b-ead5986764cc","Type":"ContainerDied","Data":"3fb4bfa15e0d3e345d0cd2f6f5f7f8b03793fa461ba39b105bf1431998d5306b"} Jul 14 00:44:54.672381 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:54.672359 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:54.672813 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:54.672437 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:54.672813 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:54.672442 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:54.672813 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:54.672524 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:54.672813 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:54.672555 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:54.672813 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:54.672607 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:54.819657 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:54.819632 2557 generic.go:356] "Generic (PLEG): container finished" podID="afc9e28a-2764-4e91-884b-ead5986764cc" containerID="454a3aedd36b326c28fb7e90a5ca0f05a50f0066b5ed3d3324f7ab34f7783d72" exitCode=0 Jul 14 00:44:54.819787 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:54.819703 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njjqg" event={"ID":"afc9e28a-2764-4e91-884b-ead5986764cc","Type":"ContainerDied","Data":"454a3aedd36b326c28fb7e90a5ca0f05a50f0066b5ed3d3324f7ab34f7783d72"} Jul 14 00:44:56.672820 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:56.672640 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:56.673339 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:56.672655 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:56.673339 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:56.672911 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:56.673339 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:56.672668 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:56.673339 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:56.672986 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:56.673339 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:56.673035 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:57.253333 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:57.253300 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:57.253606 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:57.253376 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:57.253606 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:57.253473 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:57.253606 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:57.253486 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:57.253606 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:57.253549 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs podName:4f92c479-1507-4bd3-a7a3-0bcdb89a049f nodeName:}" failed. No retries permitted until 2026-07-14 00:45:29.253525132 +0000 UTC m=+66.104892500 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs") pod "network-metrics-daemon-txzdm" (UID: "4f92c479-1507-4bd3-a7a3-0bcdb89a049f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 00:44:57.253606 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:57.253569 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret podName:5d75eb52-5424-4c0e-a6bc-1fe4d642587a nodeName:}" failed. No retries permitted until 2026-07-14 00:45:29.253560077 +0000 UTC m=+66.104927425 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret") pod "global-pull-secret-syncer-nzt74" (UID: "5d75eb52-5424-4c0e-a6bc-1fe4d642587a") : object "kube-system"/"original-pull-secret" not registered Jul 14 00:44:57.454660 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:57.454633 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmjsd\" (UniqueName: \"kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd\") pod \"network-check-target-tzdlx\" (UID: \"38610aec-8e4e-492e-8f63-1e34067176d0\") " pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:57.454821 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:57.454771 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 00:44:57.454821 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:57.454792 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 00:44:57.454821 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:57.454805 2557 projected.go:196] Error preparing data for projected volume kube-api-access-gmjsd for pod openshift-network-diagnostics/network-check-target-tzdlx: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:57.454977 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:57.454864 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd podName:38610aec-8e4e-492e-8f63-1e34067176d0 nodeName:}" failed. No retries permitted until 2026-07-14 00:45:29.454847894 +0000 UTC m=+66.306215247 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gmjsd" (UniqueName: "kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd") pod "network-check-target-tzdlx" (UID: "38610aec-8e4e-492e-8f63-1e34067176d0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 00:44:58.672550 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:58.672520 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:44:58.672927 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:58.672632 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:44:58.672927 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:58.672638 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nzt74" podUID="5d75eb52-5424-4c0e-a6bc-1fe4d642587a" Jul 14 00:44:58.672927 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:58.672718 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-tzdlx" podUID="38610aec-8e4e-492e-8f63-1e34067176d0" Jul 14 00:44:58.672927 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:58.672740 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:44:58.672927 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:58.672855 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:44:58.970822 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:58.970763 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-143.ec2.internal" event="NodeReady" Jul 14 00:44:58.970932 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:58.970875 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 00:44:59.015166 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.015122 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7n47x"] Jul 14 00:44:59.032618 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.032601 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7n47x" Jul 14 00:44:59.032713 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.032613 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-74hmz"] Jul 14 00:44:59.034468 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.034337 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 00:44:59.034468 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.034364 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 00:44:59.034468 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.034416 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 00:44:59.034468 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.034465 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rd24t\"" Jul 14 00:44:59.054841 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.054819 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7n47x"] Jul 14 00:44:59.054915 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.054845 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-74hmz"] Jul 14 00:44:59.054948 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.054922 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.056662 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.056649 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 00:44:59.056823 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.056806 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 00:44:59.057071 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.057058 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rvlnc\"" Jul 14 00:44:59.132454 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.132433 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:59.165714 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.165696 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck9gm\" (UniqueName: \"kubernetes.io/projected/95b6216a-8104-4322-960f-f53bd026111f-kube-api-access-ck9gm\") pod \"ingress-canary-7n47x\" (UID: \"95b6216a-8104-4322-960f-f53bd026111f\") " pod="openshift-ingress-canary/ingress-canary-7n47x" Jul 14 00:44:59.165794 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.165720 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p65w\" (UniqueName: \"kubernetes.io/projected/47b07188-e3fa-4065-8026-9715a5b6cb41-kube-api-access-7p65w\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.165794 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.165747 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47b07188-e3fa-4065-8026-9715a5b6cb41-tmp-dir\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.165861 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.165840 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95b6216a-8104-4322-960f-f53bd026111f-cert\") pod \"ingress-canary-7n47x\" (UID: \"95b6216a-8104-4322-960f-f53bd026111f\") " pod="openshift-ingress-canary/ingress-canary-7n47x" Jul 14 00:44:59.165893 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.165874 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47b07188-e3fa-4065-8026-9715a5b6cb41-config-volume\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.165922 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.165892 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.190242 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.190218 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:44:59.266536 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.266517 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95b6216a-8104-4322-960f-f53bd026111f-cert\") pod \"ingress-canary-7n47x\" (UID: \"95b6216a-8104-4322-960f-f53bd026111f\") " pod="openshift-ingress-canary/ingress-canary-7n47x" Jul 14 00:44:59.266596 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.266547 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47b07188-e3fa-4065-8026-9715a5b6cb41-config-volume\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.266596 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.266561 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.266596 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.266588 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck9gm\" (UniqueName: \"kubernetes.io/projected/95b6216a-8104-4322-960f-f53bd026111f-kube-api-access-ck9gm\") pod \"ingress-canary-7n47x\" (UID: \"95b6216a-8104-4322-960f-f53bd026111f\") " pod="openshift-ingress-canary/ingress-canary-7n47x" Jul 14 00:44:59.266746 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:59.266730 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 00:44:59.266799 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:59.266787 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls podName:47b07188-e3fa-4065-8026-9715a5b6cb41 nodeName:}" failed. No retries permitted until 2026-07-14 00:44:59.766770654 +0000 UTC m=+36.618138005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls") pod "dns-default-74hmz" (UID: "47b07188-e3fa-4065-8026-9715a5b6cb41") : secret "dns-default-metrics-tls" not found Jul 14 00:44:59.266855 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.266827 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p65w\" (UniqueName: \"kubernetes.io/projected/47b07188-e3fa-4065-8026-9715a5b6cb41-kube-api-access-7p65w\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.266908 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.266882 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47b07188-e3fa-4065-8026-9715a5b6cb41-tmp-dir\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.267137 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.267116 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/47b07188-e3fa-4065-8026-9715a5b6cb41-config-volume\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.267221 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.267137 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/47b07188-e3fa-4065-8026-9715a5b6cb41-tmp-dir\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.271336 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.271314 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/95b6216a-8104-4322-960f-f53bd026111f-cert\") pod \"ingress-canary-7n47x\" (UID: \"95b6216a-8104-4322-960f-f53bd026111f\") " pod="openshift-ingress-canary/ingress-canary-7n47x" Jul 14 00:44:59.274556 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.274533 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck9gm\" (UniqueName: \"kubernetes.io/projected/95b6216a-8104-4322-960f-f53bd026111f-kube-api-access-ck9gm\") pod \"ingress-canary-7n47x\" (UID: \"95b6216a-8104-4322-960f-f53bd026111f\") " pod="openshift-ingress-canary/ingress-canary-7n47x" Jul 14 00:44:59.274788 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.274773 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p65w\" (UniqueName: \"kubernetes.io/projected/47b07188-e3fa-4065-8026-9715a5b6cb41-kube-api-access-7p65w\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.341915 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.341897 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7n47x" Jul 14 00:44:59.474076 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.474050 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7n47x"] Jul 14 00:44:59.478170 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:44:59.478001 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b6216a_8104_4322_960f_f53bd026111f.slice/crio-351837e75e2276d2a3f5baa499f612c303edfbc6b94c7a5afe08a0a56fe24e1e WatchSource:0}: Error finding container 351837e75e2276d2a3f5baa499f612c303edfbc6b94c7a5afe08a0a56fe24e1e: Status 404 returned error can't find the container with id 351837e75e2276d2a3f5baa499f612c303edfbc6b94c7a5afe08a0a56fe24e1e Jul 14 00:44:59.770619 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.770562 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:44:59.770912 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:59.770671 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 00:44:59.770912 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:44:59.770725 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls podName:47b07188-e3fa-4065-8026-9715a5b6cb41 nodeName:}" failed. No retries permitted until 2026-07-14 00:45:00.770710151 +0000 UTC m=+37.622077498 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls") pod "dns-default-74hmz" (UID: "47b07188-e3fa-4065-8026-9715a5b6cb41") : secret "dns-default-metrics-tls" not found Jul 14 00:44:59.833040 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.833017 2557 generic.go:356] "Generic (PLEG): container finished" podID="afc9e28a-2764-4e91-884b-ead5986764cc" containerID="e7b9dd14f803638a804910b7fad4d65c3141366fa88f59f504a71d79ae6a6748" exitCode=0 Jul 14 00:44:59.833145 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.833099 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njjqg" event={"ID":"afc9e28a-2764-4e91-884b-ead5986764cc","Type":"ContainerDied","Data":"e7b9dd14f803638a804910b7fad4d65c3141366fa88f59f504a71d79ae6a6748"} Jul 14 00:44:59.834312 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:44:59.834293 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7n47x" event={"ID":"95b6216a-8104-4322-960f-f53bd026111f","Type":"ContainerStarted","Data":"351837e75e2276d2a3f5baa499f612c303edfbc6b94c7a5afe08a0a56fe24e1e"} Jul 14 00:45:00.672227 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.672195 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:45:00.672384 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.672237 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:45:00.672384 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.672237 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:45:00.674821 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.674792 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 00:45:00.674950 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.674837 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 00:45:00.674950 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.674895 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cmcmt\"" Jul 14 00:45:00.675048 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.674946 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tbhhd\"" Jul 14 00:45:00.675099 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.675059 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 00:45:00.675195 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.675179 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 00:45:00.779420 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.779398 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:45:00.779694 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:00.779501 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 00:45:00.779694 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:00.779581 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls podName:47b07188-e3fa-4065-8026-9715a5b6cb41 nodeName:}" failed. No retries permitted until 2026-07-14 00:45:02.779566959 +0000 UTC m=+39.630934310 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls") pod "dns-default-74hmz" (UID: "47b07188-e3fa-4065-8026-9715a5b6cb41") : secret "dns-default-metrics-tls" not found Jul 14 00:45:00.840015 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.839988 2557 generic.go:356] "Generic (PLEG): container finished" podID="afc9e28a-2764-4e91-884b-ead5986764cc" containerID="49550966290cc3fd07af310463f78060f37a6441064809148461d7974b9fdb3d" exitCode=0 Jul 14 00:45:00.840123 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:00.840042 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njjqg" event={"ID":"afc9e28a-2764-4e91-884b-ead5986764cc","Type":"ContainerDied","Data":"49550966290cc3fd07af310463f78060f37a6441064809148461d7974b9fdb3d"} Jul 14 00:45:01.845426 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:01.845403 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-njjqg" event={"ID":"afc9e28a-2764-4e91-884b-ead5986764cc","Type":"ContainerStarted","Data":"f2e9e52b60176c17764077be063bdf8af20d66a5676c3b6086c2e9358f7b3156"} Jul 14 00:45:01.865850 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:01.865814 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-njjqg" podStartSLOduration=7.518020361 podStartE2EDuration="37.865801569s" totalImagesPullingTime="30.347781208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:45:01.86521132 +0000 UTC m=+38.716578691" watchObservedRunningTime="2026-07-14 00:45:01.865801569 +0000 UTC m=+38.717168939" Jul 14 00:45:02.014082 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:02.014046 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:45:02.066007 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:02.065978 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:45:02.794965 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:02.794936 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:45:02.795096 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:02.795047 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 00:45:02.795143 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:02.795110 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls podName:47b07188-e3fa-4065-8026-9715a5b6cb41 nodeName:}" failed. No retries permitted until 2026-07-14 00:45:06.795095853 +0000 UTC m=+43.646463202 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls") pod "dns-default-74hmz" (UID: "47b07188-e3fa-4065-8026-9715a5b6cb41") : secret "dns-default-metrics-tls" not found Jul 14 00:45:02.849045 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:02.849012 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7n47x" event={"ID":"95b6216a-8104-4322-960f-f53bd026111f","Type":"ContainerStarted","Data":"e2d2c97368e7e4d537be9c9aa0c144fa1f3d78e4c4f78718a8baa23b2dce3595"} Jul 14 00:45:02.862795 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:02.862755 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7n47x" podStartSLOduration=2.500782795 podStartE2EDuration="4.862741885s" totalImagesPullingTime="2.36195909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:45:02.862390471 +0000 UTC m=+39.713757841" watchObservedRunningTime="2026-07-14 00:45:02.862741885 +0000 UTC m=+39.714109258" Jul 14 00:45:02.988418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:02.988380 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:45:06.822960 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:06.822923 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:45:06.823342 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:06.823040 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 00:45:06.823342 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:06.823091 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls podName:47b07188-e3fa-4065-8026-9715a5b6cb41 nodeName:}" failed. No retries permitted until 2026-07-14 00:45:14.823077808 +0000 UTC m=+51.674445159 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls") pod "dns-default-74hmz" (UID: "47b07188-e3fa-4065-8026-9715a5b6cb41") : secret "dns-default-metrics-tls" not found Jul 14 00:45:07.298666 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:07.298632 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:45:07.352993 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:07.352966 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:45:12.929636 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:12.929591 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:45:12.995887 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:12.995856 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:45:14.871817 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:14.871778 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:45:14.872252 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:14.871900 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 00:45:14.872252 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:14.871968 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls podName:47b07188-e3fa-4065-8026-9715a5b6cb41 nodeName:}" failed. No retries permitted until 2026-07-14 00:45:30.871952027 +0000 UTC m=+67.723319375 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls") pod "dns-default-74hmz" (UID: "47b07188-e3fa-4065-8026-9715a5b6cb41") : secret "dns-default-metrics-tls" not found Jul 14 00:45:23.229637 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:23.229594 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:45:23.824023 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:23.823997 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-v9t68" Jul 14 00:45:23.946030 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:23.946004 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:45:24.001112 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:24.001083 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:45:24.056800 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:24.056776 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:45:29.272887 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.272847 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:45:29.273374 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.272911 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:45:29.274748 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.274723 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 00:45:29.274806 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.274726 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 00:45:29.283138 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:29.283112 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 00:45:29.283252 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:29.283237 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs podName:4f92c479-1507-4bd3-a7a3-0bcdb89a049f nodeName:}" failed. No retries permitted until 2026-07-14 00:46:33.283213717 +0000 UTC m=+130.134581064 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs") pod "network-metrics-daemon-txzdm" (UID: "4f92c479-1507-4bd3-a7a3-0bcdb89a049f") : secret "metrics-daemon-secret" not found Jul 14 00:45:29.287349 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.287331 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5d75eb52-5424-4c0e-a6bc-1fe4d642587a-original-pull-secret\") pod \"global-pull-secret-syncer-nzt74\" (UID: \"5d75eb52-5424-4c0e-a6bc-1fe4d642587a\") " pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:45:29.473862 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.473821 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmjsd\" (UniqueName: \"kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd\") pod \"network-check-target-tzdlx\" (UID: \"38610aec-8e4e-492e-8f63-1e34067176d0\") " pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:45:29.475790 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.475760 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 00:45:29.483230 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.483208 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nzt74" Jul 14 00:45:29.485715 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.485696 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 00:45:29.497636 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.497615 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmjsd\" (UniqueName: \"kubernetes.io/projected/38610aec-8e4e-492e-8f63-1e34067176d0-kube-api-access-gmjsd\") pod \"network-check-target-tzdlx\" (UID: \"38610aec-8e4e-492e-8f63-1e34067176d0\") " pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:45:29.603842 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.603816 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nzt74"] Jul 14 00:45:29.607193 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:45:29.607166 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d75eb52_5424_4c0e_a6bc_1fe4d642587a.slice/crio-8338e544e84ad58c6f894e69460a4fa7b2147b15a90b52e62ff17ba7b385a119 WatchSource:0}: Error finding container 8338e544e84ad58c6f894e69460a4fa7b2147b15a90b52e62ff17ba7b385a119: Status 404 returned error can't find the container with id 8338e544e84ad58c6f894e69460a4fa7b2147b15a90b52e62ff17ba7b385a119 Jul 14 00:45:29.792100 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.792048 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tbhhd\"" Jul 14 00:45:29.801048 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.801029 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:45:29.905448 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.905416 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nzt74" event={"ID":"5d75eb52-5424-4c0e-a6bc-1fe4d642587a","Type":"ContainerStarted","Data":"8338e544e84ad58c6f894e69460a4fa7b2147b15a90b52e62ff17ba7b385a119"} Jul 14 00:45:29.915422 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:29.915397 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-tzdlx"] Jul 14 00:45:29.918446 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:45:29.918425 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38610aec_8e4e_492e_8f63_1e34067176d0.slice/crio-54a5e29bd74ab2a08a52f362246cb926ca4ff3c2faf6973fee5c0f095f87cd39 WatchSource:0}: Error finding container 54a5e29bd74ab2a08a52f362246cb926ca4ff3c2faf6973fee5c0f095f87cd39: Status 404 returned error can't find the container with id 54a5e29bd74ab2a08a52f362246cb926ca4ff3c2faf6973fee5c0f095f87cd39 Jul 14 00:45:30.884235 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:30.884204 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:45:30.884668 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:30.884346 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 00:45:30.884668 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:45:30.884415 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls podName:47b07188-e3fa-4065-8026-9715a5b6cb41 nodeName:}" failed. No retries permitted until 2026-07-14 00:46:02.884394955 +0000 UTC m=+99.735762324 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls") pod "dns-default-74hmz" (UID: "47b07188-e3fa-4065-8026-9715a5b6cb41") : secret "dns-default-metrics-tls" not found Jul 14 00:45:30.910027 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:30.909986 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tzdlx" event={"ID":"38610aec-8e4e-492e-8f63-1e34067176d0","Type":"ContainerStarted","Data":"54a5e29bd74ab2a08a52f362246cb926ca4ff3c2faf6973fee5c0f095f87cd39"} Jul 14 00:45:35.923513 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:35.923479 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nzt74" event={"ID":"5d75eb52-5424-4c0e-a6bc-1fe4d642587a","Type":"ContainerStarted","Data":"75ab6033b74a0643bfe4746e6261ab81dbcd4baf65d0b4445fb7418cace73207"} Jul 14 00:45:35.924804 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:35.924784 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-tzdlx" event={"ID":"38610aec-8e4e-492e-8f63-1e34067176d0","Type":"ContainerStarted","Data":"eb9d7eb50bfe2dae7c50143a8c8965893a91059e446ff9ef72751f7167e24ae0"} Jul 14 00:45:35.924919 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:35.924873 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:45:35.938877 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:35.938834 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nzt74" podStartSLOduration=66.64559235 podStartE2EDuration="1m11.938822737s" totalImagesPullingTime="5.293230387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:45:35.938138797 +0000 UTC m=+72.789506167" watchObservedRunningTime="2026-07-14 00:45:35.938822737 +0000 UTC m=+72.790190107" Jul 14 00:45:35.950648 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:35.950511 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-tzdlx" podStartSLOduration=66.980985969 podStartE2EDuration="1m11.950496144s" totalImagesPullingTime="4.969510175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:45:35.950418318 +0000 UTC m=+72.801785689" watchObservedRunningTime="2026-07-14 00:45:35.950496144 +0000 UTC m=+72.801863516" Jul 14 00:45:44.226669 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.226637 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r"] Jul 14 00:45:44.228424 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.228406 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" Jul 14 00:45:44.230617 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.230594 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 14 00:45:44.230617 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.230607 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 00:45:44.230785 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.230594 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 14 00:45:44.230785 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.230599 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 14 00:45:44.230785 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.230608 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2vqk4\"" Jul 14 00:45:44.241308 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.241290 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r"] Jul 14 00:45:44.261602 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.261582 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf"] Jul 14 00:45:44.263591 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.263576 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:44.265975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.265960 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 14 00:45:44.270439 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.270420 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e14359aa-6636-4334-afc4-61d8514b4e66-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5697ff889d-l9k9r\" (UID: \"e14359aa-6636-4334-afc4-61d8514b4e66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" Jul 14 00:45:44.270610 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.270583 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsvfz\" (UniqueName: \"kubernetes.io/projected/e14359aa-6636-4334-afc4-61d8514b4e66-kube-api-access-qsvfz\") pod \"managed-serviceaccount-addon-agent-5697ff889d-l9k9r\" (UID: \"e14359aa-6636-4334-afc4-61d8514b4e66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" Jul 14 00:45:44.271284 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.271268 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf"] Jul 14 00:45:44.371387 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.371365 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc646\" (UniqueName: \"kubernetes.io/projected/fa134288-a92c-4550-a735-29592aabcadf-kube-api-access-gc646\") pod \"klusterlet-addon-workmgr-78b66bb4bd-4f7sf\" (UID: \"fa134288-a92c-4550-a735-29592aabcadf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:44.371522 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.371402 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e14359aa-6636-4334-afc4-61d8514b4e66-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5697ff889d-l9k9r\" (UID: \"e14359aa-6636-4334-afc4-61d8514b4e66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" Jul 14 00:45:44.371522 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.371426 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsvfz\" (UniqueName: \"kubernetes.io/projected/e14359aa-6636-4334-afc4-61d8514b4e66-kube-api-access-qsvfz\") pod \"managed-serviceaccount-addon-agent-5697ff889d-l9k9r\" (UID: \"e14359aa-6636-4334-afc4-61d8514b4e66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" Jul 14 00:45:44.371522 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.371444 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa134288-a92c-4550-a735-29592aabcadf-tmp\") pod \"klusterlet-addon-workmgr-78b66bb4bd-4f7sf\" (UID: \"fa134288-a92c-4550-a735-29592aabcadf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:44.371522 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.371474 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fa134288-a92c-4550-a735-29592aabcadf-klusterlet-config\") pod \"klusterlet-addon-workmgr-78b66bb4bd-4f7sf\" (UID: \"fa134288-a92c-4550-a735-29592aabcadf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:44.373845 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.373823 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e14359aa-6636-4334-afc4-61d8514b4e66-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5697ff889d-l9k9r\" (UID: \"e14359aa-6636-4334-afc4-61d8514b4e66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" Jul 14 00:45:44.379429 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.379409 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsvfz\" (UniqueName: \"kubernetes.io/projected/e14359aa-6636-4334-afc4-61d8514b4e66-kube-api-access-qsvfz\") pod \"managed-serviceaccount-addon-agent-5697ff889d-l9k9r\" (UID: \"e14359aa-6636-4334-afc4-61d8514b4e66\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" Jul 14 00:45:44.472609 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.472590 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fa134288-a92c-4550-a735-29592aabcadf-klusterlet-config\") pod \"klusterlet-addon-workmgr-78b66bb4bd-4f7sf\" (UID: \"fa134288-a92c-4550-a735-29592aabcadf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:44.472714 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.472635 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc646\" (UniqueName: \"kubernetes.io/projected/fa134288-a92c-4550-a735-29592aabcadf-kube-api-access-gc646\") pod \"klusterlet-addon-workmgr-78b66bb4bd-4f7sf\" (UID: \"fa134288-a92c-4550-a735-29592aabcadf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:44.472714 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.472657 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa134288-a92c-4550-a735-29592aabcadf-tmp\") pod \"klusterlet-addon-workmgr-78b66bb4bd-4f7sf\" (UID: \"fa134288-a92c-4550-a735-29592aabcadf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:44.472967 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.472947 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa134288-a92c-4550-a735-29592aabcadf-tmp\") pod \"klusterlet-addon-workmgr-78b66bb4bd-4f7sf\" (UID: \"fa134288-a92c-4550-a735-29592aabcadf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:44.474892 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.474873 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/fa134288-a92c-4550-a735-29592aabcadf-klusterlet-config\") pod \"klusterlet-addon-workmgr-78b66bb4bd-4f7sf\" (UID: \"fa134288-a92c-4550-a735-29592aabcadf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:44.479937 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.479854 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc646\" (UniqueName: \"kubernetes.io/projected/fa134288-a92c-4550-a735-29592aabcadf-kube-api-access-gc646\") pod \"klusterlet-addon-workmgr-78b66bb4bd-4f7sf\" (UID: \"fa134288-a92c-4550-a735-29592aabcadf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:44.549937 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.549917 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" Jul 14 00:45:44.572506 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.572490 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:44.676003 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.675978 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r"] Jul 14 00:45:44.681805 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:45:44.681775 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode14359aa_6636_4334_afc4_61d8514b4e66.slice/crio-4e826234b2b62b0044ec64ea8c01d5d393ed4c4dad7db76167675fbf5bcaaf27 WatchSource:0}: Error finding container 4e826234b2b62b0044ec64ea8c01d5d393ed4c4dad7db76167675fbf5bcaaf27: Status 404 returned error can't find the container with id 4e826234b2b62b0044ec64ea8c01d5d393ed4c4dad7db76167675fbf5bcaaf27 Jul 14 00:45:44.693288 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.693267 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf"] Jul 14 00:45:44.696773 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:45:44.696747 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa134288_a92c_4550_a735_29592aabcadf.slice/crio-8ddff201afbf7ff872fd6285150c314ef8fda1709abb8aa7487fe602245ebc82 WatchSource:0}: Error finding container 8ddff201afbf7ff872fd6285150c314ef8fda1709abb8aa7487fe602245ebc82: Status 404 returned error can't find the container with id 8ddff201afbf7ff872fd6285150c314ef8fda1709abb8aa7487fe602245ebc82 Jul 14 00:45:44.947382 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.947345 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" event={"ID":"fa134288-a92c-4550-a735-29592aabcadf","Type":"ContainerStarted","Data":"8ddff201afbf7ff872fd6285150c314ef8fda1709abb8aa7487fe602245ebc82"} Jul 14 00:45:44.948354 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:44.948329 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" event={"ID":"e14359aa-6636-4334-afc4-61d8514b4e66","Type":"ContainerStarted","Data":"4e826234b2b62b0044ec64ea8c01d5d393ed4c4dad7db76167675fbf5bcaaf27"} Jul 14 00:45:47.958029 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:47.957996 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" event={"ID":"e14359aa-6636-4334-afc4-61d8514b4e66","Type":"ContainerStarted","Data":"270a72b02e4135f05025e98323352c8f328d197c7fb97d74b3ef43aeebcaaee8"} Jul 14 00:45:47.972178 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:47.972116 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" podStartSLOduration=1.407728729 podStartE2EDuration="3.972102018s" totalImagesPullingTime="2.564373289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:45:47.970579025 +0000 UTC m=+84.821946399" watchObservedRunningTime="2026-07-14 00:45:47.972102018 +0000 UTC m=+84.823469389" Jul 14 00:45:50.968586 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:50.968515 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" event={"ID":"fa134288-a92c-4550-a735-29592aabcadf","Type":"ContainerStarted","Data":"69425ea1223d671f5b3e56c8c24980e1f5deae87bb03764d517b73e7c03597df"} Jul 14 00:45:50.968905 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:50.968714 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:50.970069 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:50.970049 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:45:50.983877 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:45:50.983833 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" podStartSLOduration=1.074130043 podStartE2EDuration="6.983818958s" totalImagesPullingTime="5.909688915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:45:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:45:50.982808281 +0000 UTC m=+87.834175664" watchObservedRunningTime="2026-07-14 00:45:50.983818958 +0000 UTC m=+87.835186327" Jul 14 00:46:02.895175 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:46:02.895103 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:46:02.895711 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:46:02.895287 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 00:46:02.895711 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:46:02.895397 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls podName:47b07188-e3fa-4065-8026-9715a5b6cb41 nodeName:}" failed. No retries permitted until 2026-07-14 00:47:06.895373302 +0000 UTC m=+163.746740655 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls") pod "dns-default-74hmz" (UID: "47b07188-e3fa-4065-8026-9715a5b6cb41") : secret "dns-default-metrics-tls" not found Jul 14 00:46:06.930163 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:46:06.930121 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-tzdlx" Jul 14 00:46:33.285453 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:46:33.285416 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:46:33.285922 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:46:33.285556 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 00:46:33.285922 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:46:33.285608 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs podName:4f92c479-1507-4bd3-a7a3-0bcdb89a049f nodeName:}" failed. No retries permitted until 2026-07-14 00:48:35.285595731 +0000 UTC m=+252.136963078 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs") pod "network-metrics-daemon-txzdm" (UID: "4f92c479-1507-4bd3-a7a3-0bcdb89a049f") : secret "metrics-daemon-secret" not found Jul 14 00:47:02.062477 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:47:02.062444 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-74hmz" podUID="47b07188-e3fa-4065-8026-9715a5b6cb41" Jul 14 00:47:03.696334 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:47:03.696297 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-txzdm" podUID="4f92c479-1507-4bd3-a7a3-0bcdb89a049f" Jul 14 00:47:06.905949 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:06.905913 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:47:06.906456 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:47:06.906062 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 00:47:06.906456 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:47:06.906133 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls podName:47b07188-e3fa-4065-8026-9715a5b6cb41 nodeName:}" failed. No retries permitted until 2026-07-14 00:49:08.90611652 +0000 UTC m=+285.757483867 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls") pod "dns-default-74hmz" (UID: "47b07188-e3fa-4065-8026-9715a5b6cb41") : secret "dns-default-metrics-tls" not found Jul 14 00:47:15.672684 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:15.672648 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:47:17.672646 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:17.672609 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-74hmz" Jul 14 00:47:43.771027 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:43.770998 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:47:48.293929 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:48.293897 2557 generic.go:356] "Generic (PLEG): container finished" podID="e14359aa-6636-4334-afc4-61d8514b4e66" containerID="270a72b02e4135f05025e98323352c8f328d197c7fb97d74b3ef43aeebcaaee8" exitCode=255 Jul 14 00:47:48.294408 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:48.293953 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" event={"ID":"e14359aa-6636-4334-afc4-61d8514b4e66","Type":"ContainerDied","Data":"270a72b02e4135f05025e98323352c8f328d197c7fb97d74b3ef43aeebcaaee8"} Jul 14 00:47:48.299311 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:48.299291 2557 scope.go:117] "RemoveContainer" containerID="270a72b02e4135f05025e98323352c8f328d197c7fb97d74b3ef43aeebcaaee8" Jul 14 00:47:49.297706 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:49.297667 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5697ff889d-l9k9r" event={"ID":"e14359aa-6636-4334-afc4-61d8514b4e66","Type":"ContainerStarted","Data":"b7639a32f663b777d0e03c2a4644578ea6cb21b7d0102d7163a3ddc617426235"} Jul 14 00:47:50.969201 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:50.969093 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" podUID="fa134288-a92c-4550-a735-29592aabcadf" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.9:8000/readyz\": dial tcp 10.134.0.9:8000: connect: connection refused" Jul 14 00:47:51.304755 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:51.304722 2557 generic.go:356] "Generic (PLEG): container finished" podID="fa134288-a92c-4550-a735-29592aabcadf" containerID="69425ea1223d671f5b3e56c8c24980e1f5deae87bb03764d517b73e7c03597df" exitCode=1 Jul 14 00:47:51.304888 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:51.304772 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" event={"ID":"fa134288-a92c-4550-a735-29592aabcadf","Type":"ContainerDied","Data":"69425ea1223d671f5b3e56c8c24980e1f5deae87bb03764d517b73e7c03597df"} Jul 14 00:47:51.305062 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:51.305050 2557 scope.go:117] "RemoveContainer" containerID="69425ea1223d671f5b3e56c8c24980e1f5deae87bb03764d517b73e7c03597df" Jul 14 00:47:52.309975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:52.309942 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" event={"ID":"fa134288-a92c-4550-a735-29592aabcadf","Type":"ContainerStarted","Data":"fd132735880fed2832fb25501d1f87f13860ee80e7197701e3fc4e32bdd29f50"} Jul 14 00:47:52.310362 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:52.310277 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:47:52.310866 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:47:52.310846 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78b66bb4bd-4f7sf" Jul 14 00:48:12.621282 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.621241 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5cf98649f6-l5t6k"] Jul 14 00:48:12.623315 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.623292 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.625990 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.625964 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 14 00:48:12.627588 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.627567 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 14 00:48:12.627689 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.627573 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 14 00:48:12.628716 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.628688 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-np425\"" Jul 14 00:48:12.633914 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.633892 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 14 00:48:12.638677 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.638653 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cf98649f6-l5t6k"] Jul 14 00:48:12.707028 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.707003 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-installation-pull-secrets\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.707117 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.707030 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-bound-sa-token\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.707117 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.707088 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-image-registry-private-configuration\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.707220 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.707124 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.707220 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.707141 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-certificates\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.707220 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.707207 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da29410b-0ffa-410f-a741-0f7c68f21f4c-ca-trust-extracted\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.707312 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.707227 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlkql\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-kube-api-access-qlkql\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.707312 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.707243 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-trusted-ca\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.713707 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.713688 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hbcm5"] Jul 14 00:48:12.718513 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.718486 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.720170 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.720117 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-blkvf\"" Jul 14 00:48:12.720360 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.720282 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 00:48:12.720360 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.720282 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 14 00:48:12.720542 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.720523 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 14 00:48:12.720979 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.720958 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 00:48:12.726699 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.726668 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 14 00:48:12.728460 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.728439 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hbcm5"] Jul 14 00:48:12.808085 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808063 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c824f147-b37c-41d9-a199-3f1399bb93d5-snapshots\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.808205 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808092 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-image-registry-private-configuration\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.808205 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808112 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.808205 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-certificates\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.808205 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808177 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c824f147-b37c-41d9-a199-3f1399bb93d5-tmp\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.808415 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:12.808243 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 00:48:12.808415 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:12.808262 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cf98649f6-l5t6k: secret "image-registry-tls" not found Jul 14 00:48:12.808415 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808317 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c824f147-b37c-41d9-a199-3f1399bb93d5-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.808415 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:12.808351 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls podName:da29410b-0ffa-410f-a741-0f7c68f21f4c nodeName:}" failed. No retries permitted until 2026-07-14 00:48:13.30832585 +0000 UTC m=+230.159693207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls") pod "image-registry-5cf98649f6-l5t6k" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c") : secret "image-registry-tls" not found Jul 14 00:48:12.808415 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808398 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da29410b-0ffa-410f-a741-0f7c68f21f4c-ca-trust-extracted\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.808659 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808421 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlkql\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-kube-api-access-qlkql\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.808659 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808439 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdxkf\" (UniqueName: \"kubernetes.io/projected/c824f147-b37c-41d9-a199-3f1399bb93d5-kube-api-access-tdxkf\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.808659 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808549 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-trusted-ca\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.808659 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808586 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-installation-pull-secrets\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.808659 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808627 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-bound-sa-token\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.808892 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808752 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-certificates\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.808892 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808759 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c824f147-b37c-41d9-a199-3f1399bb93d5-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.808892 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808773 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da29410b-0ffa-410f-a741-0f7c68f21f4c-ca-trust-extracted\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.808892 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.808788 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c824f147-b37c-41d9-a199-3f1399bb93d5-serving-cert\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.809313 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.809294 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-trusted-ca\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.810761 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.810742 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-image-registry-private-configuration\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.810945 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.810928 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-installation-pull-secrets\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.816585 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.816559 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlkql\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-kube-api-access-qlkql\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.820394 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.820376 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-bound-sa-token\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:12.865365 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.865337 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:12.909556 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.909489 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c824f147-b37c-41d9-a199-3f1399bb93d5-snapshots\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.909556 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.909531 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c824f147-b37c-41d9-a199-3f1399bb93d5-tmp\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.909556 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.909549 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c824f147-b37c-41d9-a199-3f1399bb93d5-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.909714 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.909578 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdxkf\" (UniqueName: \"kubernetes.io/projected/c824f147-b37c-41d9-a199-3f1399bb93d5-kube-api-access-tdxkf\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.909714 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.909630 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c824f147-b37c-41d9-a199-3f1399bb93d5-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.909714 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.909652 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c824f147-b37c-41d9-a199-3f1399bb93d5-serving-cert\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.910107 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.910088 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c824f147-b37c-41d9-a199-3f1399bb93d5-tmp\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.910196 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.910181 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c824f147-b37c-41d9-a199-3f1399bb93d5-snapshots\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.910305 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.910221 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c824f147-b37c-41d9-a199-3f1399bb93d5-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.910458 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.910438 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c824f147-b37c-41d9-a199-3f1399bb93d5-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.911634 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.911609 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c824f147-b37c-41d9-a199-3f1399bb93d5-serving-cert\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.916592 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.916571 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdxkf\" (UniqueName: \"kubernetes.io/projected/c824f147-b37c-41d9-a199-3f1399bb93d5-kube-api-access-tdxkf\") pod \"insights-operator-68d9dd94cf-hbcm5\" (UID: \"c824f147-b37c-41d9-a199-3f1399bb93d5\") " pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:12.961413 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:12.961389 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:13.028288 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:13.028268 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" Jul 14 00:48:13.140780 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:13.140749 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-hbcm5"] Jul 14 00:48:13.143936 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:48:13.143910 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc824f147_b37c_41d9_a199_3f1399bb93d5.slice/crio-3fd35e611e6344c5a63fdaa3e3c2c88416bc861a93b9fa722533916c4aa735bd WatchSource:0}: Error finding container 3fd35e611e6344c5a63fdaa3e3c2c88416bc861a93b9fa722533916c4aa735bd: Status 404 returned error can't find the container with id 3fd35e611e6344c5a63fdaa3e3c2c88416bc861a93b9fa722533916c4aa735bd Jul 14 00:48:13.311694 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:13.311663 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:13.311791 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:13.311774 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 00:48:13.311791 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:13.311786 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cf98649f6-l5t6k: secret "image-registry-tls" not found Jul 14 00:48:13.311865 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:13.311829 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls podName:da29410b-0ffa-410f-a741-0f7c68f21f4c nodeName:}" failed. No retries permitted until 2026-07-14 00:48:14.311814704 +0000 UTC m=+231.163182052 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls") pod "image-registry-5cf98649f6-l5t6k" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c") : secret "image-registry-tls" not found Jul 14 00:48:13.372346 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:13.372317 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" event={"ID":"c824f147-b37c-41d9-a199-3f1399bb93d5","Type":"ContainerStarted","Data":"3fd35e611e6344c5a63fdaa3e3c2c88416bc861a93b9fa722533916c4aa735bd"} Jul 14 00:48:14.318267 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:14.318240 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:14.318669 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:14.318345 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 00:48:14.318669 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:14.318358 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cf98649f6-l5t6k: secret "image-registry-tls" not found Jul 14 00:48:14.318669 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:14.318409 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls podName:da29410b-0ffa-410f-a741-0f7c68f21f4c nodeName:}" failed. No retries permitted until 2026-07-14 00:48:16.318394109 +0000 UTC m=+233.169761458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls") pod "image-registry-5cf98649f6-l5t6k" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c") : secret "image-registry-tls" not found Jul 14 00:48:15.456945 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:15.456912 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:15.558128 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:15.558094 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:16.332369 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:16.332337 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:16.332510 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:16.332476 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 00:48:16.332510 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:16.332494 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cf98649f6-l5t6k: secret "image-registry-tls" not found Jul 14 00:48:16.332582 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:16.332541 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls podName:da29410b-0ffa-410f-a741-0f7c68f21f4c nodeName:}" failed. No retries permitted until 2026-07-14 00:48:20.332527483 +0000 UTC m=+237.183894859 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls") pod "image-registry-5cf98649f6-l5t6k" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c") : secret "image-registry-tls" not found Jul 14 00:48:16.382195 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:16.382169 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" event={"ID":"c824f147-b37c-41d9-a199-3f1399bb93d5","Type":"ContainerStarted","Data":"f180dd477e774ca4c51246163233b1c994ba156a65c4a5b2ac415cc652b64b78"} Jul 14 00:48:16.396634 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:16.396590 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" podStartSLOduration=1.896551492 podStartE2EDuration="4.396578689s" totalImagesPullingTime="2.500027197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:48:16.39571814 +0000 UTC m=+233.247085510" watchObservedRunningTime="2026-07-14 00:48:16.396578689 +0000 UTC m=+233.247946059" Jul 14 00:48:16.968688 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:16.968660 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m7464"] Jul 14 00:48:16.970825 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:16.970799 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:16.972573 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:16.972552 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 00:48:16.972661 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:16.972552 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 00:48:16.972661 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:16.972633 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-942xj\"" Jul 14 00:48:16.980114 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:16.980094 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m7464"] Jul 14 00:48:17.003992 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.003974 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:17.038248 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.038226 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pg2s\" (UniqueName: \"kubernetes.io/projected/847c344a-b356-4dbc-8143-d7f3154fdf3a-kube-api-access-5pg2s\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.038320 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.038278 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/847c344a-b356-4dbc-8143-d7f3154fdf3a-crio-socket\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.038320 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.038294 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/847c344a-b356-4dbc-8143-d7f3154fdf3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.038398 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.038350 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/847c344a-b356-4dbc-8143-d7f3154fdf3a-data-volume\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.038398 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.038373 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.057558 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.057538 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:17.139098 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.139074 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/847c344a-b356-4dbc-8143-d7f3154fdf3a-crio-socket\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.139211 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.139099 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/847c344a-b356-4dbc-8143-d7f3154fdf3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.139211 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.139122 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/847c344a-b356-4dbc-8143-d7f3154fdf3a-data-volume\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.139211 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.139140 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.139211 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.139201 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pg2s\" (UniqueName: \"kubernetes.io/projected/847c344a-b356-4dbc-8143-d7f3154fdf3a-kube-api-access-5pg2s\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.139211 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.139204 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/847c344a-b356-4dbc-8143-d7f3154fdf3a-crio-socket\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.139411 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:17.139221 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 00:48:17.139411 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:17.139281 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/847c344a-b356-4dbc-8143-d7f3154fdf3a-kube-rbac-proxy-cm podName:847c344a-b356-4dbc-8143-d7f3154fdf3a nodeName:}" failed. No retries permitted until 2026-07-14 00:48:17.639266944 +0000 UTC m=+234.490634291 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/847c344a-b356-4dbc-8143-d7f3154fdf3a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m7464" (UID: "847c344a-b356-4dbc-8143-d7f3154fdf3a") : configmap "kube-rbac-proxy" not found Jul 14 00:48:17.139411 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:17.139283 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 00:48:17.139411 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:17.139323 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls podName:847c344a-b356-4dbc-8143-d7f3154fdf3a nodeName:}" failed. No retries permitted until 2026-07-14 00:48:17.639310264 +0000 UTC m=+234.490677612 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m7464" (UID: "847c344a-b356-4dbc-8143-d7f3154fdf3a") : secret "insights-runtime-extractor-tls" not found Jul 14 00:48:17.139646 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.139534 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/847c344a-b356-4dbc-8143-d7f3154fdf3a-data-volume\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.147183 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.147137 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pg2s\" (UniqueName: \"kubernetes.io/projected/847c344a-b356-4dbc-8143-d7f3154fdf3a-kube-api-access-5pg2s\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.309969 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.309949 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:17.643351 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.643299 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/847c344a-b356-4dbc-8143-d7f3154fdf3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.643351 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.643336 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.643469 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:17.643430 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 00:48:17.643501 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:17.643472 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls podName:847c344a-b356-4dbc-8143-d7f3154fdf3a nodeName:}" failed. No retries permitted until 2026-07-14 00:48:18.643461331 +0000 UTC m=+235.494828679 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m7464" (UID: "847c344a-b356-4dbc-8143-d7f3154fdf3a") : secret "insights-runtime-extractor-tls" not found Jul 14 00:48:17.643732 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.643715 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/847c344a-b356-4dbc-8143-d7f3154fdf3a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:17.731546 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.731528 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:17.832198 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.832170 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:17.932736 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:17.932686 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:18.153062 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:18.153034 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:18.208098 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:18.208035 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:18.426485 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:18.426455 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:18.531750 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:18.531691 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:18.650193 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:18.650170 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:18.650295 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:18.650281 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 00:48:18.650347 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:18.650337 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls podName:847c344a-b356-4dbc-8143-d7f3154fdf3a nodeName:}" failed. No retries permitted until 2026-07-14 00:48:20.650321745 +0000 UTC m=+237.501689094 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m7464" (UID: "847c344a-b356-4dbc-8143-d7f3154fdf3a") : secret "insights-runtime-extractor-tls" not found Jul 14 00:48:18.753869 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:18.753850 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7xct5_c8dacd2e-6336-4956-aa9f-e9a0964534d0/dns-node-resolver/0.log" Jul 14 00:48:18.834866 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:18.834830 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:18.897094 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:18.897073 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:19.953957 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:19.953935 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qqpzf_427427dc-5248-4fdb-b49f-1309f98f2320/node-ca/0.log" Jul 14 00:48:20.026112 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.026086 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:20.039893 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.039863 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qgdbt"] Jul 14 00:48:20.041985 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.041969 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" Jul 14 00:48:20.044044 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.044021 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 14 00:48:20.044180 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.044047 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 14 00:48:20.044384 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.044363 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-c5n4q\"" Jul 14 00:48:20.044449 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.044390 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 14 00:48:20.044449 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.044364 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 14 00:48:20.052591 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.052569 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qgdbt"] Jul 14 00:48:20.059761 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.059740 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rjs6\" (UniqueName: \"kubernetes.io/projected/f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048-kube-api-access-7rjs6\") pod \"service-ca-79f68b9f4f-qgdbt\" (UID: \"f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" Jul 14 00:48:20.059863 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.059778 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048-signing-key\") pod \"service-ca-79f68b9f4f-qgdbt\" (UID: \"f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" Jul 14 00:48:20.059863 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.059803 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048-signing-cabundle\") pod \"service-ca-79f68b9f4f-qgdbt\" (UID: \"f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" Jul 14 00:48:20.088448 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.088424 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:20.160796 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.160773 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rjs6\" (UniqueName: \"kubernetes.io/projected/f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048-kube-api-access-7rjs6\") pod \"service-ca-79f68b9f4f-qgdbt\" (UID: \"f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" Jul 14 00:48:20.160880 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.160812 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048-signing-key\") pod \"service-ca-79f68b9f4f-qgdbt\" (UID: \"f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" Jul 14 00:48:20.160880 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.160844 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048-signing-cabundle\") pod \"service-ca-79f68b9f4f-qgdbt\" (UID: \"f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" Jul 14 00:48:20.161576 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.161558 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048-signing-cabundle\") pod \"service-ca-79f68b9f4f-qgdbt\" (UID: \"f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" Jul 14 00:48:20.163204 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.163185 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048-signing-key\") pod \"service-ca-79f68b9f4f-qgdbt\" (UID: \"f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" Jul 14 00:48:20.167436 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.167404 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rjs6\" (UniqueName: \"kubernetes.io/projected/f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048-kube-api-access-7rjs6\") pod \"service-ca-79f68b9f4f-qgdbt\" (UID: \"f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048\") " pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" Jul 14 00:48:20.352021 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.351998 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" Jul 14 00:48:20.363114 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.363089 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:20.363271 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:20.363251 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 14 00:48:20.363357 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:20.363275 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5cf98649f6-l5t6k: secret "image-registry-tls" not found Jul 14 00:48:20.363357 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:20.363335 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls podName:da29410b-0ffa-410f-a741-0f7c68f21f4c nodeName:}" failed. No retries permitted until 2026-07-14 00:48:28.363314653 +0000 UTC m=+245.214682004 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls") pod "image-registry-5cf98649f6-l5t6k" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c") : secret "image-registry-tls" not found Jul 14 00:48:20.370187 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.370163 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:20.473111 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.473085 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-qgdbt"] Jul 14 00:48:20.475990 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:48:20.475969 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf20567e4_2ce8_43c8_b1c8_4e6cbc6fe048.slice/crio-1c937997a492691e6a5dec48e7d73a370152334c0bec67f860119dd42028fd07 WatchSource:0}: Error finding container 1c937997a492691e6a5dec48e7d73a370152334c0bec67f860119dd42028fd07: Status 404 returned error can't find the container with id 1c937997a492691e6a5dec48e7d73a370152334c0bec67f860119dd42028fd07 Jul 14 00:48:20.554841 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.554824 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7n47x_95b6216a-8104-4322-960f-f53bd026111f/serve-healthcheck-canary/0.log" Jul 14 00:48:20.666279 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.666225 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:20.666362 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:20.666338 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 00:48:20.666403 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:20.666382 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls podName:847c344a-b356-4dbc-8143-d7f3154fdf3a nodeName:}" failed. No retries permitted until 2026-07-14 00:48:24.666370777 +0000 UTC m=+241.517738128 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m7464" (UID: "847c344a-b356-4dbc-8143-d7f3154fdf3a") : secret "insights-runtime-extractor-tls" not found Jul 14 00:48:20.786175 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:20.786136 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:21.398555 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:21.398526 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" event={"ID":"f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048","Type":"ContainerStarted","Data":"1c937997a492691e6a5dec48e7d73a370152334c0bec67f860119dd42028fd07"} Jul 14 00:48:23.072482 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:23.072437 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:23.238712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:23.238676 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:23.338787 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:23.338696 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:23.406882 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:23.406825 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" event={"ID":"f20567e4-2ce8-43c8-b1c8-4e6cbc6fe048","Type":"ContainerStarted","Data":"510f51be5b38f74839dd9442e7c6b30f7d565651d77e732a2a0fc04c4ab65156"} Jul 14 00:48:23.427277 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:23.427231 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-qgdbt" podStartSLOduration=1.487548616 podStartE2EDuration="3.427214812s" totalImagesPullingTime="1.939666196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:48:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:48:23.426359965 +0000 UTC m=+240.277727337" watchObservedRunningTime="2026-07-14 00:48:23.427214812 +0000 UTC m=+240.278582188" Jul 14 00:48:23.434251 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:23.434219 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:23.508259 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:23.508237 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:23.646406 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:23.646347 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:23.802117 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:23.802089 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:23.862724 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:23.862704 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:24.696847 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:24.696815 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:24.697256 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:24.696950 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 00:48:24.697256 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:24.697014 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls podName:847c344a-b356-4dbc-8143-d7f3154fdf3a nodeName:}" failed. No retries permitted until 2026-07-14 00:48:32.696997018 +0000 UTC m=+249.548364371 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m7464" (UID: "847c344a-b356-4dbc-8143-d7f3154fdf3a") : secret "insights-runtime-extractor-tls" not found Jul 14 00:48:24.786632 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:24.786604 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:24.982709 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:24.982654 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:25.362761 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:25.362736 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:25.572356 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:25.572337 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:25.771312 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:25.771247 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:26.765168 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:26.765119 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:26.953007 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:26.952982 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:27.008132 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:27.008109 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:27.159922 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:27.159900 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:27.372415 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:27.372395 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:27.561476 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:27.561455 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:27.760599 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:27.760579 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:28.421033 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:28.421010 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:28.423452 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:28.423428 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls\") pod \"image-registry-5cf98649f6-l5t6k\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:28.534897 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:28.534873 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-np425\"" Jul 14 00:48:28.543408 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:28.543390 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:28.567901 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:28.567875 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:28.664777 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:28.664751 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5cf98649f6-l5t6k"] Jul 14 00:48:28.667680 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:48:28.667652 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda29410b_0ffa_410f_a741_0f7c68f21f4c.slice/crio-3060d41c82e515815025b32626afb6e91fcdb3e919a9d7d6f150b1517783ad22 WatchSource:0}: Error finding container 3060d41c82e515815025b32626afb6e91fcdb3e919a9d7d6f150b1517783ad22: Status 404 returned error can't find the container with id 3060d41c82e515815025b32626afb6e91fcdb3e919a9d7d6f150b1517783ad22 Jul 14 00:48:29.170393 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:29.170360 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:29.368094 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:29.368063 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:29.425080 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:29.425015 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" event={"ID":"da29410b-0ffa-410f-a741-0f7c68f21f4c","Type":"ContainerStarted","Data":"7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf"} Jul 14 00:48:29.425080 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:29.425063 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" event={"ID":"da29410b-0ffa-410f-a741-0f7c68f21f4c","Type":"ContainerStarted","Data":"3060d41c82e515815025b32626afb6e91fcdb3e919a9d7d6f150b1517783ad22"} Jul 14 00:48:29.425519 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:29.425136 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:48:29.457006 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:29.456959 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" podStartSLOduration=17.456942138 podStartE2EDuration="17.456942138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:48:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:48:29.456711322 +0000 UTC m=+246.308078693" watchObservedRunningTime="2026-07-14 00:48:29.456942138 +0000 UTC m=+246.308309509" Jul 14 00:48:29.636892 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:29.636860 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:29.761732 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:29.761679 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:29.960090 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:29.960070 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:30.177254 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:30.177222 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:30.961183 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:30.961137 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:31.570177 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:31.570133 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:31.761520 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:31.761484 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:31.964124 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:31.964063 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:32.140367 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:32.140342 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:32.361467 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:32.361441 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:32.750910 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:32.750830 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:32.753035 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:32.753000 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/847c344a-b356-4dbc-8143-d7f3154fdf3a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m7464\" (UID: \"847c344a-b356-4dbc-8143-d7f3154fdf3a\") " pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:32.882115 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:32.882087 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-942xj\"" Jul 14 00:48:32.890460 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:32.890444 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m7464" Jul 14 00:48:32.963002 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:32.962973 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:33.019399 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:33.019335 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m7464"] Jul 14 00:48:33.022339 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:48:33.022312 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847c344a_b356_4dbc_8143_d7f3154fdf3a.slice/crio-c2ee35c2f31e288ff4fd6facf335074325cbbccc83a8492910c8151fd7f8de4b WatchSource:0}: Error finding container c2ee35c2f31e288ff4fd6facf335074325cbbccc83a8492910c8151fd7f8de4b: Status 404 returned error can't find the container with id c2ee35c2f31e288ff4fd6facf335074325cbbccc83a8492910c8151fd7f8de4b Jul 14 00:48:33.197237 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:33.197212 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:33.437808 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:33.437772 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m7464" event={"ID":"847c344a-b356-4dbc-8143-d7f3154fdf3a","Type":"ContainerStarted","Data":"c2ee35c2f31e288ff4fd6facf335074325cbbccc83a8492910c8151fd7f8de4b"} Jul 14 00:48:33.563847 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:33.563774 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:33.763178 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:33.763011 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:35.017139 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:35.017099 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:35.076911 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:35.076877 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:35.371692 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:35.371657 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:48:35.374310 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:35.374283 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f92c479-1507-4bd3-a7a3-0bcdb89a049f-metrics-certs\") pod \"network-metrics-daemon-txzdm\" (UID: \"4f92c479-1507-4bd3-a7a3-0bcdb89a049f\") " pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:48:35.474821 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:35.474782 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cmcmt\"" Jul 14 00:48:35.483131 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:35.483087 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-txzdm" Jul 14 00:48:36.790557 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:36.790535 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-txzdm"] Jul 14 00:48:36.793452 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:48:36.793420 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f92c479_1507_4bd3_a7a3_0bcdb89a049f.slice/crio-cdc1ed66de0754f3853f12477f225ce28c1a5274694bd8bb1366fb47ca009b43 WatchSource:0}: Error finding container cdc1ed66de0754f3853f12477f225ce28c1a5274694bd8bb1366fb47ca009b43: Status 404 returned error can't find the container with id cdc1ed66de0754f3853f12477f225ce28c1a5274694bd8bb1366fb47ca009b43 Jul 14 00:48:37.453964 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:37.453931 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m7464" event={"ID":"847c344a-b356-4dbc-8143-d7f3154fdf3a","Type":"ContainerStarted","Data":"fad17b59b127261ea5d652d8738abe118ab0f76882725f827d363b8b33cab9af"} Jul 14 00:48:37.455269 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:37.455241 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-txzdm" event={"ID":"4f92c479-1507-4bd3-a7a3-0bcdb89a049f","Type":"ContainerStarted","Data":"cdc1ed66de0754f3853f12477f225ce28c1a5274694bd8bb1366fb47ca009b43"} Jul 14 00:48:37.560849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:37.560817 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:37.662398 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:37.662361 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:37.741927 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:37.741892 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:37.862472 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:37.862452 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:38.047056 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:38.047031 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:38.241357 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:38.241320 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:38.364796 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:38.364708 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:38.460453 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:38.460420 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m7464" event={"ID":"847c344a-b356-4dbc-8143-d7f3154fdf3a","Type":"ContainerStarted","Data":"da467b9110c59fc9a535bbe91deab88d8ececc968020cc7d09331dde0492ac9d"} Jul 14 00:48:38.462411 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:38.462243 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-txzdm" event={"ID":"4f92c479-1507-4bd3-a7a3-0bcdb89a049f","Type":"ContainerStarted","Data":"1be5bcd351dc04a5cc59bed1d4d1e59e245c15f5ad77aad26e14a437470dde44"} Jul 14 00:48:38.462411 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:38.462300 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-txzdm" event={"ID":"4f92c479-1507-4bd3-a7a3-0bcdb89a049f","Type":"ContainerStarted","Data":"a9bf232a47bdfa81e16f94a66e484bf8f0075fd459994d56f5a7f234833313d4"} Jul 14 00:48:38.477363 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:38.477309 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-txzdm" podStartSLOduration=253.545993276 podStartE2EDuration="4m14.477291902s" totalImagesPullingTime="931.298626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:44:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:48:38.476716664 +0000 UTC m=+255.328084035" watchObservedRunningTime="2026-07-14 00:48:38.477291902 +0000 UTC m=+255.328659273" Jul 14 00:48:38.640252 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:38.640185 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:38.739636 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:38.739590 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:38.840978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:38.840934 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:38.895536 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:38.895476 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:39.257016 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.256933 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:39.475129 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.475098 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-lpkgf"] Jul 14 00:48:39.481285 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.481261 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cf98649f6-l5t6k"] Jul 14 00:48:39.481434 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.481416 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-lpkgf" Jul 14 00:48:39.486594 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.484316 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 00:48:39.486594 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.484645 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 00:48:39.486594 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.484964 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-lcx5j\"" Jul 14 00:48:39.490727 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.490701 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-lpkgf"] Jul 14 00:48:39.517753 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.517704 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6f6b674cc4-wxvnx"] Jul 14 00:48:39.522430 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.522411 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.534630 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.534609 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f6b674cc4-wxvnx"] Jul 14 00:48:39.564355 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.564329 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:39.606472 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.606444 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5020942f-9d6a-400f-8714-6da764b5e6ea-ca-trust-extracted\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.606582 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.606476 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5020942f-9d6a-400f-8714-6da764b5e6ea-image-registry-private-configuration\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.606582 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.606502 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5020942f-9d6a-400f-8714-6da764b5e6ea-trusted-ca\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.606707 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.606640 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mwhx\" (UniqueName: \"kubernetes.io/projected/f0354a6b-116e-4fa2-b001-0dd0434a2ec1-kube-api-access-2mwhx\") pod \"downloads-68c9cbb497-lpkgf\" (UID: \"f0354a6b-116e-4fa2-b001-0dd0434a2ec1\") " pod="openshift-console/downloads-68c9cbb497-lpkgf" Jul 14 00:48:39.606707 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.606667 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5020942f-9d6a-400f-8714-6da764b5e6ea-bound-sa-token\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.606815 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.606714 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5020942f-9d6a-400f-8714-6da764b5e6ea-installation-pull-secrets\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.606815 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.606778 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5020942f-9d6a-400f-8714-6da764b5e6ea-registry-tls\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.606815 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.606803 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5020942f-9d6a-400f-8714-6da764b5e6ea-registry-certificates\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.606969 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.606837 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2db8d\" (UniqueName: \"kubernetes.io/projected/5020942f-9d6a-400f-8714-6da764b5e6ea-kube-api-access-2db8d\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.657725 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.657649 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:39.707779 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.707751 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2db8d\" (UniqueName: \"kubernetes.io/projected/5020942f-9d6a-400f-8714-6da764b5e6ea-kube-api-access-2db8d\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.707897 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.707803 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5020942f-9d6a-400f-8714-6da764b5e6ea-ca-trust-extracted\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.708019 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.707982 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5020942f-9d6a-400f-8714-6da764b5e6ea-image-registry-private-configuration\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.708140 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.708037 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5020942f-9d6a-400f-8714-6da764b5e6ea-trusted-ca\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.708140 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.708093 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mwhx\" (UniqueName: \"kubernetes.io/projected/f0354a6b-116e-4fa2-b001-0dd0434a2ec1-kube-api-access-2mwhx\") pod \"downloads-68c9cbb497-lpkgf\" (UID: \"f0354a6b-116e-4fa2-b001-0dd0434a2ec1\") " pod="openshift-console/downloads-68c9cbb497-lpkgf" Jul 14 00:48:39.708140 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.708129 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5020942f-9d6a-400f-8714-6da764b5e6ea-bound-sa-token\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.708336 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.708175 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5020942f-9d6a-400f-8714-6da764b5e6ea-installation-pull-secrets\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.708336 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.708202 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5020942f-9d6a-400f-8714-6da764b5e6ea-registry-tls\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.708336 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.708231 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5020942f-9d6a-400f-8714-6da764b5e6ea-registry-certificates\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.708479 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.708355 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5020942f-9d6a-400f-8714-6da764b5e6ea-ca-trust-extracted\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.709035 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.708991 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5020942f-9d6a-400f-8714-6da764b5e6ea-registry-certificates\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.709139 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.709084 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5020942f-9d6a-400f-8714-6da764b5e6ea-trusted-ca\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.711103 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.711079 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5020942f-9d6a-400f-8714-6da764b5e6ea-image-registry-private-configuration\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.711103 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.711100 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5020942f-9d6a-400f-8714-6da764b5e6ea-registry-tls\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.711273 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.711127 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5020942f-9d6a-400f-8714-6da764b5e6ea-installation-pull-secrets\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.716903 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.716880 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2db8d\" (UniqueName: \"kubernetes.io/projected/5020942f-9d6a-400f-8714-6da764b5e6ea-kube-api-access-2db8d\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.719694 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.719661 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5020942f-9d6a-400f-8714-6da764b5e6ea-bound-sa-token\") pod \"image-registry-6f6b674cc4-wxvnx\" (UID: \"5020942f-9d6a-400f-8714-6da764b5e6ea\") " pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.719778 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.719668 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mwhx\" (UniqueName: \"kubernetes.io/projected/f0354a6b-116e-4fa2-b001-0dd0434a2ec1-kube-api-access-2mwhx\") pod \"downloads-68c9cbb497-lpkgf\" (UID: \"f0354a6b-116e-4fa2-b001-0dd0434a2ec1\") " pod="openshift-console/downloads-68c9cbb497-lpkgf" Jul 14 00:48:39.758759 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.758727 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:39.796053 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.796033 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-lpkgf" Jul 14 00:48:39.833746 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.833720 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:39.856926 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:39.856901 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:40.279342 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:40.279313 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:40.326636 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:40.326612 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-lpkgf"] Jul 14 00:48:40.331128 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:48:40.331106 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0354a6b_116e_4fa2_b001_0dd0434a2ec1.slice/crio-0f2d14b46a7cdcd8922000e44198eee4b40c205dec217eb450d6bd164162cbfd WatchSource:0}: Error finding container 0f2d14b46a7cdcd8922000e44198eee4b40c205dec217eb450d6bd164162cbfd: Status 404 returned error can't find the container with id 0f2d14b46a7cdcd8922000e44198eee4b40c205dec217eb450d6bd164162cbfd Jul 14 00:48:40.342680 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:40.342659 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6f6b674cc4-wxvnx"] Jul 14 00:48:40.345498 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:48:40.345477 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5020942f_9d6a_400f_8714_6da764b5e6ea.slice/crio-a1c6a9e151078580bb5e760b2699f42e03de9714174ed3cb829b73a6d26280b5 WatchSource:0}: Error finding container a1c6a9e151078580bb5e760b2699f42e03de9714174ed3cb829b73a6d26280b5: Status 404 returned error can't find the container with id a1c6a9e151078580bb5e760b2699f42e03de9714174ed3cb829b73a6d26280b5 Jul 14 00:48:40.470343 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:40.470287 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" event={"ID":"5020942f-9d6a-400f-8714-6da764b5e6ea","Type":"ContainerStarted","Data":"93985d9be8858ba3d398555400f94dfcff4713998875a30bff0c05451fa6febb"} Jul 14 00:48:40.470343 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:40.470329 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" event={"ID":"5020942f-9d6a-400f-8714-6da764b5e6ea","Type":"ContainerStarted","Data":"a1c6a9e151078580bb5e760b2699f42e03de9714174ed3cb829b73a6d26280b5"} Jul 14 00:48:40.470512 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:40.470365 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:48:40.471503 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:40.471476 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-lpkgf" event={"ID":"f0354a6b-116e-4fa2-b001-0dd0434a2ec1","Type":"ContainerStarted","Data":"0f2d14b46a7cdcd8922000e44198eee4b40c205dec217eb450d6bd164162cbfd"} Jul 14 00:48:40.473625 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:40.473603 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m7464" event={"ID":"847c344a-b356-4dbc-8143-d7f3154fdf3a","Type":"ContainerStarted","Data":"f8272fae11dbbe5df7b3d6e8551ed486978747c746e1ab1bdf254bacb93e76ec"} Jul 14 00:48:40.487994 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:40.487953 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" podStartSLOduration=1.487937912 podStartE2EDuration="1.487937912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:48:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:48:40.486861586 +0000 UTC m=+257.338228955" watchObservedRunningTime="2026-07-14 00:48:40.487937912 +0000 UTC m=+257.339305281" Jul 14 00:48:40.502929 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:40.502884 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m7464" podStartSLOduration=17.423556775 podStartE2EDuration="24.502871123s" totalImagesPullingTime="7.079314348s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 00:48:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:48:40.501514553 +0000 UTC m=+257.352881923" watchObservedRunningTime="2026-07-14 00:48:40.502871123 +0000 UTC m=+257.354238492" Jul 14 00:48:47.176284 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:47.176199 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:47.278750 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:47.278719 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:47.336073 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:47.336013 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:47.645812 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:47.645785 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:48.866590 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:48.866550 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:48.916913 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:48.916858 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:49.487900 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:49.487864 2557 patch_prober.go:28] interesting pod/image-registry-5cf98649f6-l5t6k container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 00:48:49.488064 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:49.487918 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" podUID="da29410b-0ffa-410f-a741-0f7c68f21f4c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 00:48:50.020134 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:50.020100 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:50.070592 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:50.070559 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:51.062905 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:51.062864 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:51.355323 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:51.355249 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:51.416300 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:51.416259 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:51.894859 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:51.894824 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:53.920161 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:53.920128 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:53.977934 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:53.977908 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:54.484997 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:54.484955 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:56.498139 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.498108 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:56.602028 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.601985 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:56.716611 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.716569 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:56.755607 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.755506 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-28k6t"] Jul 14 00:48:56.762112 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.762091 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-5dpnm"] Jul 14 00:48:56.762265 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.762245 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.764354 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.764334 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-2zq6p\"" Jul 14 00:48:56.764842 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.764393 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 00:48:56.764842 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.764429 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 14 00:48:56.764842 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.764708 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 14 00:48:56.764842 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.764723 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 00:48:56.765097 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.764927 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 00:48:56.765097 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.765055 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 14 00:48:56.766372 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.766353 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.770377 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.768013 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 00:48:56.770377 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.768235 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 00:48:56.770377 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.768355 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6nrc5\"" Jul 14 00:48:56.770377 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.768610 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 00:48:56.770377 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.770144 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-28k6t"] Jul 14 00:48:56.807759 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.807731 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:56.834420 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834393 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vn87t\" (UniqueName: \"kubernetes.io/projected/081b053a-2396-4f00-8f3d-e867400806bb-kube-api-access-vn87t\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.834518 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834426 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/081b053a-2396-4f00-8f3d-e867400806bb-metrics-client-ca\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.834518 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834458 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.834656 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834523 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-textfile\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.834656 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834553 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/081b053a-2396-4f00-8f3d-e867400806bb-root\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.834656 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834576 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-tls\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.834656 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834627 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.834855 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834659 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/081b053a-2396-4f00-8f3d-e867400806bb-sys\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.834855 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834682 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.834855 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834743 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.834855 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834769 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-accelerators-collector-config\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.834855 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834797 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mzmc\" (UniqueName: \"kubernetes.io/projected/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-api-access-8mzmc\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.834855 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834825 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-wtmp\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.834855 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834851 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.835207 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.834901 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.877223 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.877197 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:56.935276 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935252 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vn87t\" (UniqueName: \"kubernetes.io/projected/081b053a-2396-4f00-8f3d-e867400806bb-kube-api-access-vn87t\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.935391 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935291 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/081b053a-2396-4f00-8f3d-e867400806bb-metrics-client-ca\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.935391 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935314 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.935391 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935346 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-textfile\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.935572 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935549 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/081b053a-2396-4f00-8f3d-e867400806bb-root\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.935632 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935607 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-tls\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.935681 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935653 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.935681 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935665 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-textfile\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.935777 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935677 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/081b053a-2396-4f00-8f3d-e867400806bb-root\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.935777 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935681 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/081b053a-2396-4f00-8f3d-e867400806bb-sys\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.935777 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935721 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.935777 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935735 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/081b053a-2396-4f00-8f3d-e867400806bb-sys\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.935777 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935764 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.936020 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935789 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-accelerators-collector-config\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.936020 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935822 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mzmc\" (UniqueName: \"kubernetes.io/projected/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-api-access-8mzmc\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.936020 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935852 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-wtmp\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.936020 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935875 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.936020 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935927 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.936020 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.935942 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/081b053a-2396-4f00-8f3d-e867400806bb-metrics-client-ca\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.936324 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.936020 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.936324 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:56.936118 2557 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 14 00:48:56.936324 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.936190 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-wtmp\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.936324 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:48:56.936210 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-tls podName:0daf5dc4-d3ee-4414-8858-3e83d3ba2abf nodeName:}" failed. No retries permitted until 2026-07-14 00:48:57.436190965 +0000 UTC m=+274.287558316 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-28k6t" (UID: "0daf5dc4-d3ee-4414-8858-3e83d3ba2abf") : secret "kube-state-metrics-tls" not found Jul 14 00:48:56.936324 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.936254 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.936697 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.936674 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-accelerators-collector-config\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.936798 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.936772 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.938890 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.938814 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.939106 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.939044 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/081b053a-2396-4f00-8f3d-e867400806bb-node-exporter-tls\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.939513 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.939474 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.947413 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.947373 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vn87t\" (UniqueName: \"kubernetes.io/projected/081b053a-2396-4f00-8f3d-e867400806bb-kube-api-access-vn87t\") pod \"node-exporter-5dpnm\" (UID: \"081b053a-2396-4f00-8f3d-e867400806bb\") " pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:56.947555 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.947514 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mzmc\" (UniqueName: \"kubernetes.io/projected/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-api-access-8mzmc\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:56.982454 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:56.982423 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:57.080536 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:57.080497 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:57.083485 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:57.083466 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-5dpnm" Jul 14 00:48:57.197587 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:57.197558 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:57.439330 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:57.439245 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:57.441872 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:57.441842 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0daf5dc4-d3ee-4414-8858-3e83d3ba2abf-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-28k6t\" (UID: \"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:57.677871 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:57.677846 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" Jul 14 00:48:57.721360 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:57.721289 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:57.807600 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:57.807567 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:57.921208 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:48:57.921182 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod081b053a_2396_4f00_8f3d_e867400806bb.slice/crio-5017c2a760cbe5131755036e104aa411e506b991f188f0452a89b179256b2424 WatchSource:0}: Error finding container 5017c2a760cbe5131755036e104aa411e506b991f188f0452a89b179256b2424: Status 404 returned error can't find the container with id 5017c2a760cbe5131755036e104aa411e506b991f188f0452a89b179256b2424 Jul 14 00:48:57.925516 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:57.925488 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:58.036343 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:58.036317 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-28k6t"] Jul 14 00:48:58.045083 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:48:58.045054 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0daf5dc4_d3ee_4414_8858_3e83d3ba2abf.slice/crio-a2dbbd293d806e64e708f5b2032cbe937af93088db27f10c0f0e2f241ba44046 WatchSource:0}: Error finding container a2dbbd293d806e64e708f5b2032cbe937af93088db27f10c0f0e2f241ba44046: Status 404 returned error can't find the container with id a2dbbd293d806e64e708f5b2032cbe937af93088db27f10c0f0e2f241ba44046 Jul 14 00:48:58.540092 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:58.540051 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-lpkgf" event={"ID":"f0354a6b-116e-4fa2-b001-0dd0434a2ec1","Type":"ContainerStarted","Data":"d27b43381311d6760f60d9a1d18d19a56c1d560a5a59efb908404e2763603957"} Jul 14 00:48:58.540330 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:58.540293 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-lpkgf" Jul 14 00:48:58.542418 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:58.542391 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5dpnm" event={"ID":"081b053a-2396-4f00-8f3d-e867400806bb","Type":"ContainerStarted","Data":"5017c2a760cbe5131755036e104aa411e506b991f188f0452a89b179256b2424"} Jul 14 00:48:58.543777 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:58.543756 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" event={"ID":"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf","Type":"ContainerStarted","Data":"a2dbbd293d806e64e708f5b2032cbe937af93088db27f10c0f0e2f241ba44046"} Jul 14 00:48:58.551671 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:58.551646 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-lpkgf" Jul 14 00:48:58.557780 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:58.557730 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-lpkgf" podStartSLOduration=1.8773824110000001 podStartE2EDuration="19.557711965s" totalImagesPullingTime="17.680329554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:48:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:48:58.556638729 +0000 UTC m=+275.408006099" watchObservedRunningTime="2026-07-14 00:48:58.557711965 +0000 UTC m=+275.409079335" Jul 14 00:48:59.487231 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:59.487189 2557 patch_prober.go:28] interesting pod/image-registry-5cf98649f6-l5t6k container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 00:48:59.487781 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:59.487746 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" podUID="da29410b-0ffa-410f-a741-0f7c68f21f4c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 00:48:59.551238 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:59.551201 2557 generic.go:356] "Generic (PLEG): container finished" podID="081b053a-2396-4f00-8f3d-e867400806bb" containerID="73a5397395c2a85aa97726ade2d72d18dfd7a4806f61f47b832f5612ad2f99ab" exitCode=0 Jul 14 00:48:59.551470 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:59.551301 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5dpnm" event={"ID":"081b053a-2396-4f00-8f3d-e867400806bb","Type":"ContainerDied","Data":"73a5397395c2a85aa97726ade2d72d18dfd7a4806f61f47b832f5612ad2f99ab"} Jul 14 00:48:59.674333 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:59.674297 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:59.775629 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:59.775597 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:48:59.840111 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:59.840075 2557 patch_prober.go:28] interesting pod/image-registry-6f6b674cc4-wxvnx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 00:48:59.840298 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:48:59.840135 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" podUID="5020942f-9d6a-400f-8714-6da764b5e6ea" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 00:49:00.091117 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:00.091081 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:00.345319 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:00.345289 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:00.557774 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:00.557733 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5dpnm" event={"ID":"081b053a-2396-4f00-8f3d-e867400806bb","Type":"ContainerStarted","Data":"df8c7100dac5a82ae045bcaf43066b7329d41779a30e89e99eb90e19424ee5fb"} Jul 14 00:49:00.558282 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:00.557804 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-5dpnm" event={"ID":"081b053a-2396-4f00-8f3d-e867400806bb","Type":"ContainerStarted","Data":"b2cd5001dd52163deab49551ecf3fc8956b072758848dcf71587087b9d8d3b0a"} Jul 14 00:49:00.560695 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:00.560662 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" event={"ID":"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf","Type":"ContainerStarted","Data":"7aa4583e6a08ce1b16462136ea67465e83530b808f7986c0846832d008866693"} Jul 14 00:49:00.560833 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:00.560733 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" event={"ID":"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf","Type":"ContainerStarted","Data":"bf6a7ce8a435eac234d96cc1305d5d9931743ac1be9ce531b7d296641f98edba"} Jul 14 00:49:00.560833 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:00.560771 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" event={"ID":"0daf5dc4-d3ee-4414-8858-3e83d3ba2abf","Type":"ContainerStarted","Data":"a96ada68e7b82874d479ae5ab18bfc5b4170e3410a2bb070b23edba90740a455"} Jul 14 00:49:00.602574 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:00.602076 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-5dpnm" podStartSLOduration=3.46187936 podStartE2EDuration="4.602057044s" totalImagesPullingTime="1.055370641s" totalInitContainerRuntime="84.807043ms" isStatefulPod=true podCreationTimestamp="2026-07-14 00:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:49:00.580393345 +0000 UTC m=+277.431760717" watchObservedRunningTime="2026-07-14 00:49:00.602057044 +0000 UTC m=+277.453424415" Jul 14 00:49:01.483168 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:01.483123 2557 patch_prober.go:28] interesting pod/image-registry-6f6b674cc4-wxvnx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 00:49:01.483453 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:01.483198 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" podUID="5020942f-9d6a-400f-8714-6da764b5e6ea" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 00:49:01.551584 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:01.551551 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:01.608406 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:01.608373 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:01.704689 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:01.704653 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:02.946817 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:02.946779 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:03.039867 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:03.039837 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:04.509381 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:04.509337 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" podUID="da29410b-0ffa-410f-a741-0f7c68f21f4c" containerName="registry" containerID="cri-o://7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf" gracePeriod=30 Jul 14 00:49:05.301972 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.301944 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:49:05.320494 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.320197 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-28k6t" podStartSLOduration=7.269506129 podStartE2EDuration="9.320176732s" totalImagesPullingTime="2.050670603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:49:00.601690126 +0000 UTC m=+277.453057497" watchObservedRunningTime="2026-07-14 00:49:05.320176732 +0000 UTC m=+282.171544104" Jul 14 00:49:05.409741 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.409678 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-trusted-ca\") pod \"da29410b-0ffa-410f-a741-0f7c68f21f4c\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " Jul 14 00:49:05.409741 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.409724 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-certificates\") pod \"da29410b-0ffa-410f-a741-0f7c68f21f4c\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " Jul 14 00:49:05.409916 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.409752 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qlkql\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-kube-api-access-qlkql\") pod \"da29410b-0ffa-410f-a741-0f7c68f21f4c\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " Jul 14 00:49:05.409916 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.409780 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da29410b-0ffa-410f-a741-0f7c68f21f4c-ca-trust-extracted\") pod \"da29410b-0ffa-410f-a741-0f7c68f21f4c\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " Jul 14 00:49:05.409916 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.409798 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-bound-sa-token\") pod \"da29410b-0ffa-410f-a741-0f7c68f21f4c\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " Jul 14 00:49:05.409916 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.409851 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-image-registry-private-configuration\") pod \"da29410b-0ffa-410f-a741-0f7c68f21f4c\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " Jul 14 00:49:05.409916 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.409880 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-installation-pull-secrets\") pod \"da29410b-0ffa-410f-a741-0f7c68f21f4c\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " Jul 14 00:49:05.410189 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.409931 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls\") pod \"da29410b-0ffa-410f-a741-0f7c68f21f4c\" (UID: \"da29410b-0ffa-410f-a741-0f7c68f21f4c\") " Jul 14 00:49:05.410189 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.410071 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "da29410b-0ffa-410f-a741-0f7c68f21f4c" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 00:49:05.410189 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.410095 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "da29410b-0ffa-410f-a741-0f7c68f21f4c" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 00:49:05.410344 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.410208 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-certificates\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:49:05.410344 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.410241 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/da29410b-0ffa-410f-a741-0f7c68f21f4c-trusted-ca\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:49:05.412978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.412942 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "da29410b-0ffa-410f-a741-0f7c68f21f4c" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 00:49:05.412978 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.412947 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "da29410b-0ffa-410f-a741-0f7c68f21f4c" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:49:05.413192 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.413169 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "da29410b-0ffa-410f-a741-0f7c68f21f4c" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 00:49:05.413261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.413206 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-kube-api-access-qlkql" (OuterVolumeSpecName: "kube-api-access-qlkql") pod "da29410b-0ffa-410f-a741-0f7c68f21f4c" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c"). InnerVolumeSpecName "kube-api-access-qlkql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:49:05.413317 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.413260 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "da29410b-0ffa-410f-a741-0f7c68f21f4c" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:49:05.422588 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.422563 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da29410b-0ffa-410f-a741-0f7c68f21f4c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "da29410b-0ffa-410f-a741-0f7c68f21f4c" (UID: "da29410b-0ffa-410f-a741-0f7c68f21f4c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:49:05.510948 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.510919 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-installation-pull-secrets\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:49:05.510948 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.510946 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-registry-tls\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:49:05.511315 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.510956 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlkql\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-kube-api-access-qlkql\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:49:05.511315 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.510965 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/da29410b-0ffa-410f-a741-0f7c68f21f4c-ca-trust-extracted\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:49:05.511315 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.510974 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/da29410b-0ffa-410f-a741-0f7c68f21f4c-bound-sa-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:49:05.511315 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.510988 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/da29410b-0ffa-410f-a741-0f7c68f21f4c-image-registry-private-configuration\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:49:05.588417 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.588389 2557 generic.go:356] "Generic (PLEG): container finished" podID="da29410b-0ffa-410f-a741-0f7c68f21f4c" containerID="7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf" exitCode=0 Jul 14 00:49:05.588527 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.588449 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" Jul 14 00:49:05.588527 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.588484 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" event={"ID":"da29410b-0ffa-410f-a741-0f7c68f21f4c","Type":"ContainerDied","Data":"7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf"} Jul 14 00:49:05.588639 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.588536 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5cf98649f6-l5t6k" event={"ID":"da29410b-0ffa-410f-a741-0f7c68f21f4c","Type":"ContainerDied","Data":"3060d41c82e515815025b32626afb6e91fcdb3e919a9d7d6f150b1517783ad22"} Jul 14 00:49:05.588639 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.588563 2557 scope.go:117] "RemoveContainer" containerID="7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf" Jul 14 00:49:05.598467 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.598449 2557 scope.go:117] "RemoveContainer" containerID="7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf" Jul 14 00:49:05.598748 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:49:05.598724 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf\": container with ID starting with 7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf not found: ID does not exist" containerID="7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf" Jul 14 00:49:05.598818 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.598760 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf"} err="failed to get container status \"7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf\": rpc error: code = NotFound desc = could not find container \"7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf\": container with ID starting with 7e9146c9b12ee9844c99fc1b806a2c4aefdcf6f927a884552e8479d8e33b87cf not found: ID does not exist" Jul 14 00:49:05.610685 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.610665 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5cf98649f6-l5t6k"] Jul 14 00:49:05.615774 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.615753 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5cf98649f6-l5t6k"] Jul 14 00:49:05.677745 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:05.677698 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da29410b-0ffa-410f-a741-0f7c68f21f4c" path="/var/lib/kubelet/pods/da29410b-0ffa-410f-a741-0f7c68f21f4c/volumes" Jul 14 00:49:07.280447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.280407 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:07.383036 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.382999 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:07.433763 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.433735 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:07.710254 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.710230 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9775947b5-k4jf9"] Jul 14 00:49:07.710497 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.710485 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da29410b-0ffa-410f-a741-0f7c68f21f4c" containerName="registry" Jul 14 00:49:07.710544 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.710498 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="da29410b-0ffa-410f-a741-0f7c68f21f4c" containerName="registry" Jul 14 00:49:07.710615 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.710597 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="da29410b-0ffa-410f-a741-0f7c68f21f4c" containerName="registry" Jul 14 00:49:07.761686 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.761660 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9775947b5-k4jf9"] Jul 14 00:49:07.761778 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.761736 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:07.763514 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.763497 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 00:49:07.763625 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.763540 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 00:49:07.763759 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.763743 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4t8jn\"" Jul 14 00:49:07.763759 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.763750 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 00:49:07.763964 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.763950 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 00:49:07.764296 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.764279 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 00:49:07.768404 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.768377 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 00:49:07.879098 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.879074 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:07.929459 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.929438 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-serving-cert\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:07.929552 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.929466 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-service-ca\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:07.929552 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.929497 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-console-config\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:07.929552 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.929530 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-oauth-serving-cert\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:07.929663 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.929561 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-oauth-config\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:07.929663 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.929592 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgpt4\" (UniqueName: \"kubernetes.io/projected/3361b806-60dc-4179-9af7-10d0f2796a1f-kube-api-access-pgpt4\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:07.929663 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.929618 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-trusted-ca-bundle\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:07.980438 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:07.980394 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:08.030483 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.030464 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-serving-cert\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.030575 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.030491 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-service-ca\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.030575 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.030521 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-console-config\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.030575 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.030540 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-oauth-serving-cert\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.030575 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.030556 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-oauth-config\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.030575 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.030574 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgpt4\" (UniqueName: \"kubernetes.io/projected/3361b806-60dc-4179-9af7-10d0f2796a1f-kube-api-access-pgpt4\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.030821 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.030590 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-trusted-ca-bundle\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.031297 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.031262 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-console-config\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.031396 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.031318 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-service-ca\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.031443 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.031410 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-oauth-serving-cert\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.031517 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.031497 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-trusted-ca-bundle\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.033142 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.033121 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-oauth-config\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.033235 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.033140 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-serving-cert\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.037501 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.037483 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgpt4\" (UniqueName: \"kubernetes.io/projected/3361b806-60dc-4179-9af7-10d0f2796a1f-kube-api-access-pgpt4\") pod \"console-9775947b5-k4jf9\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.071329 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.071305 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:08.209477 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.209449 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9775947b5-k4jf9"] Jul 14 00:49:08.212358 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:49:08.212329 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3361b806_60dc_4179_9af7_10d0f2796a1f.slice/crio-7fdf3de83553f6f8e5bc776c18e8553a435cc94675961efb40a39cea4861490a WatchSource:0}: Error finding container 7fdf3de83553f6f8e5bc776c18e8553a435cc94675961efb40a39cea4861490a: Status 404 returned error can't find the container with id 7fdf3de83553f6f8e5bc776c18e8553a435cc94675961efb40a39cea4861490a Jul 14 00:49:08.479670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.479645 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:08.580701 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.580673 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:08.600774 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.600749 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9775947b5-k4jf9" event={"ID":"3361b806-60dc-4179-9af7-10d0f2796a1f","Type":"ContainerStarted","Data":"7fdf3de83553f6f8e5bc776c18e8553a435cc94675961efb40a39cea4861490a"} Jul 14 00:49:08.938914 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.938885 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:49:08.941381 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.941360 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/47b07188-e3fa-4065-8026-9715a5b6cb41-metrics-tls\") pod \"dns-default-74hmz\" (UID: \"47b07188-e3fa-4065-8026-9715a5b6cb41\") " pod="openshift-dns/dns-default-74hmz" Jul 14 00:49:08.975315 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.975293 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rvlnc\"" Jul 14 00:49:08.983592 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:08.983574 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-74hmz" Jul 14 00:49:09.117565 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:09.117542 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-74hmz"] Jul 14 00:49:09.119738 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:49:09.119715 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47b07188_e3fa_4065_8026_9715a5b6cb41.slice/crio-7e982d82a7c93eb12f3c3c8e4afa9c76a8fcabaa4008324ef29784544dc769cf WatchSource:0}: Error finding container 7e982d82a7c93eb12f3c3c8e4afa9c76a8fcabaa4008324ef29784544dc769cf: Status 404 returned error can't find the container with id 7e982d82a7c93eb12f3c3c8e4afa9c76a8fcabaa4008324ef29784544dc769cf Jul 14 00:49:09.605974 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:09.605933 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-74hmz" event={"ID":"47b07188-e3fa-4065-8026-9715a5b6cb41","Type":"ContainerStarted","Data":"7e982d82a7c93eb12f3c3c8e4afa9c76a8fcabaa4008324ef29784544dc769cf"} Jul 14 00:49:09.839334 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:09.839302 2557 patch_prober.go:28] interesting pod/image-registry-6f6b674cc4-wxvnx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 14 00:49:09.839492 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:09.839355 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" podUID="5020942f-9d6a-400f-8714-6da764b5e6ea" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 00:49:11.482371 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:11.482345 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6f6b674cc4-wxvnx" Jul 14 00:49:11.600764 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:11.600725 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:12.619140 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:12.619108 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9775947b5-k4jf9" event={"ID":"3361b806-60dc-4179-9af7-10d0f2796a1f","Type":"ContainerStarted","Data":"9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e"} Jul 14 00:49:12.625072 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:12.625046 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:12.635817 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:12.635776 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9775947b5-k4jf9" podStartSLOduration=1.35835265 podStartE2EDuration="5.635758351s" totalImagesPullingTime="4.277405701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:49:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:49:12.634349186 +0000 UTC m=+289.485716550" watchObservedRunningTime="2026-07-14 00:49:12.635758351 +0000 UTC m=+289.487125721" Jul 14 00:49:12.925209 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:12.925178 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:12.974662 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:12.974594 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:13.624275 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:13.624244 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-74hmz" event={"ID":"47b07188-e3fa-4065-8026-9715a5b6cb41","Type":"ContainerStarted","Data":"eea252a1f7b910609308ad4786d5404eef60e0355d359f317ce2338c40c3bde1"} Jul 14 00:49:13.624783 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:13.624299 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-74hmz" event={"ID":"47b07188-e3fa-4065-8026-9715a5b6cb41","Type":"ContainerStarted","Data":"c9f7be96a573885754610e0426895275505f31f40620fc5c2195f59a688dc0ba"} Jul 14 00:49:13.624783 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:13.624414 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-74hmz" Jul 14 00:49:13.642324 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:13.642194 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-74hmz" podStartSLOduration=252.26708797 podStartE2EDuration="4m15.642175369s" totalImagesPullingTime="3.375087399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:49:13.641174486 +0000 UTC m=+290.492541855" watchObservedRunningTime="2026-07-14 00:49:13.642175369 +0000 UTC m=+290.493542734" Jul 14 00:49:13.921905 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:13.921843 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:14.020948 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:14.020927 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:14.143882 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:14.143854 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:14.206442 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:14.206397 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:16.134465 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:16.134433 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:16.193731 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:16.193698 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:16.618670 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:16.618642 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:18.071735 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:18.071705 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:18.072218 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:18.071762 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:18.076703 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:18.076674 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:18.643706 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:18.643672 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:49:18.799059 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:18.799029 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:18.898228 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:18.898166 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:19.004320 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:19.004294 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:20.028112 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:20.028084 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:20.128798 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:20.128776 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:21.957767 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:21.957730 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:22.034274 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:22.034243 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:23.017712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:23.017677 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:23.109918 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:23.109888 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:23.502405 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:23.502383 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 00:49:23.503954 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:23.503934 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 00:49:23.510430 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:23.510405 2557 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 00:49:23.629811 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:23.629789 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-74hmz" Jul 14 00:49:32.741442 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:32.741394 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:32.795415 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:32.795376 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:35.025645 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:35.025612 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:35.126179 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:35.126136 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:36.700938 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:36.700907 2557 generic.go:356] "Generic (PLEG): container finished" podID="c824f147-b37c-41d9-a199-3f1399bb93d5" containerID="f180dd477e774ca4c51246163233b1c994ba156a65c4a5b2ac415cc652b64b78" exitCode=0 Jul 14 00:49:36.701371 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:36.700945 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" event={"ID":"c824f147-b37c-41d9-a199-3f1399bb93d5","Type":"ContainerDied","Data":"f180dd477e774ca4c51246163233b1c994ba156a65c4a5b2ac415cc652b64b78"} Jul 14 00:49:36.701371 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:36.701315 2557 scope.go:117] "RemoveContainer" containerID="f180dd477e774ca4c51246163233b1c994ba156a65c4a5b2ac415cc652b64b78" Jul 14 00:49:37.706257 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:37.706228 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-hbcm5" event={"ID":"c824f147-b37c-41d9-a199-3f1399bb93d5","Type":"ContainerStarted","Data":"23ce286374a779103383e2d6356f48ccc75010e7e74586e31d75f79031e5d0bc"} Jul 14 00:49:50.030222 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:50.030177 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:49:50.083972 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:49:50.083941 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:12.844570 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:12.844531 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:12.892686 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:12.892650 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:23.325144 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:23.325109 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:23.387983 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:23.387953 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:24.662905 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:24.662870 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:24.764487 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:24.764459 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:25.968985 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:25.968952 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:26.029528 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:26.029502 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:35.028289 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:35.028254 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:35.129008 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:35.128975 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:37.821468 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:37.821436 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:37.875837 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:37.875810 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:37.968034 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:37.968012 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:38.015883 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:38.015857 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:38.583315 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:38.583288 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:38.632999 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:38.632979 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:48.370873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:48.370839 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9775947b5-k4jf9"] Jul 14 00:50:48.462265 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:48.462229 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:48.514680 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:48.514649 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:48.570365 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:48.570333 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:50:51.574804 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:50:51.574766 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:13.391664 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.391622 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-9775947b5-k4jf9" podUID="3361b806-60dc-4179-9af7-10d0f2796a1f" containerName="console" containerID="cri-o://9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e" gracePeriod=15 Jul 14 00:51:13.634946 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.634919 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:51:13.701045 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.700986 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-trusted-ca-bundle\") pod \"3361b806-60dc-4179-9af7-10d0f2796a1f\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " Jul 14 00:51:13.701045 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.701023 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-console-config\") pod \"3361b806-60dc-4179-9af7-10d0f2796a1f\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " Jul 14 00:51:13.701045 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.701043 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-oauth-serving-cert\") pod \"3361b806-60dc-4179-9af7-10d0f2796a1f\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " Jul 14 00:51:13.701335 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.701070 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-oauth-config\") pod \"3361b806-60dc-4179-9af7-10d0f2796a1f\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " Jul 14 00:51:13.701335 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.701091 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-service-ca\") pod \"3361b806-60dc-4179-9af7-10d0f2796a1f\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " Jul 14 00:51:13.701335 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.701130 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-serving-cert\") pod \"3361b806-60dc-4179-9af7-10d0f2796a1f\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " Jul 14 00:51:13.701335 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.701169 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgpt4\" (UniqueName: \"kubernetes.io/projected/3361b806-60dc-4179-9af7-10d0f2796a1f-kube-api-access-pgpt4\") pod \"3361b806-60dc-4179-9af7-10d0f2796a1f\" (UID: \"3361b806-60dc-4179-9af7-10d0f2796a1f\") " Jul 14 00:51:13.701546 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.701519 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3361b806-60dc-4179-9af7-10d0f2796a1f" (UID: "3361b806-60dc-4179-9af7-10d0f2796a1f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 00:51:13.701598 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.701526 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3361b806-60dc-4179-9af7-10d0f2796a1f" (UID: "3361b806-60dc-4179-9af7-10d0f2796a1f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 00:51:13.701658 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.701631 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-console-config" (OuterVolumeSpecName: "console-config") pod "3361b806-60dc-4179-9af7-10d0f2796a1f" (UID: "3361b806-60dc-4179-9af7-10d0f2796a1f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 00:51:13.701658 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.701646 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-service-ca" (OuterVolumeSpecName: "service-ca") pod "3361b806-60dc-4179-9af7-10d0f2796a1f" (UID: "3361b806-60dc-4179-9af7-10d0f2796a1f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 00:51:13.703358 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.703335 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3361b806-60dc-4179-9af7-10d0f2796a1f" (UID: "3361b806-60dc-4179-9af7-10d0f2796a1f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 00:51:13.703358 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.703347 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3361b806-60dc-4179-9af7-10d0f2796a1f" (UID: "3361b806-60dc-4179-9af7-10d0f2796a1f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 00:51:13.703503 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.703342 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3361b806-60dc-4179-9af7-10d0f2796a1f-kube-api-access-pgpt4" (OuterVolumeSpecName: "kube-api-access-pgpt4") pod "3361b806-60dc-4179-9af7-10d0f2796a1f" (UID: "3361b806-60dc-4179-9af7-10d0f2796a1f"). InnerVolumeSpecName "kube-api-access-pgpt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:51:13.802106 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.802086 2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-serving-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:51:13.802106 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.802106 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgpt4\" (UniqueName: \"kubernetes.io/projected/3361b806-60dc-4179-9af7-10d0f2796a1f-kube-api-access-pgpt4\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:51:13.802248 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.802115 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-trusted-ca-bundle\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:51:13.802248 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.802124 2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-console-config\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:51:13.802248 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.802132 2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-oauth-serving-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:51:13.802248 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.802141 2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3361b806-60dc-4179-9af7-10d0f2796a1f-console-oauth-config\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:51:13.802248 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:13.802166 2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3361b806-60dc-4179-9af7-10d0f2796a1f-service-ca\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:51:14.030477 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:14.030402 2557 generic.go:356] "Generic (PLEG): container finished" podID="3361b806-60dc-4179-9af7-10d0f2796a1f" containerID="9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e" exitCode=0 Jul 14 00:51:14.030477 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:14.030447 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9775947b5-k4jf9" event={"ID":"3361b806-60dc-4179-9af7-10d0f2796a1f","Type":"ContainerDied","Data":"9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e"} Jul 14 00:51:14.030477 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:14.030451 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9775947b5-k4jf9" Jul 14 00:51:14.030724 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:14.030487 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9775947b5-k4jf9" event={"ID":"3361b806-60dc-4179-9af7-10d0f2796a1f","Type":"ContainerDied","Data":"7fdf3de83553f6f8e5bc776c18e8553a435cc94675961efb40a39cea4861490a"} Jul 14 00:51:14.030724 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:14.030506 2557 scope.go:117] "RemoveContainer" containerID="9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e" Jul 14 00:51:14.038599 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:14.038580 2557 scope.go:117] "RemoveContainer" containerID="9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e" Jul 14 00:51:14.038842 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:51:14.038825 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e\": container with ID starting with 9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e not found: ID does not exist" containerID="9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e" Jul 14 00:51:14.038898 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:14.038849 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e"} err="failed to get container status \"9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e\": rpc error: code = NotFound desc = could not find container \"9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e\": container with ID starting with 9a3e5e328286ed245a3dcf44071c14d63d8094b8a4729df3cc5f0c43e3adf81e not found: ID does not exist" Jul 14 00:51:14.049004 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:14.048980 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9775947b5-k4jf9"] Jul 14 00:51:14.052402 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:14.052383 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9775947b5-k4jf9"] Jul 14 00:51:15.676108 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:15.676076 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3361b806-60dc-4179-9af7-10d0f2796a1f" path="/var/lib/kubelet/pods/3361b806-60dc-4179-9af7-10d0f2796a1f/volumes" Jul 14 00:51:20.996315 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:20.996284 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:21.093409 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.093382 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:21.586674 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.586627 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr"] Jul 14 00:51:21.586893 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.586879 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3361b806-60dc-4179-9af7-10d0f2796a1f" containerName="console" Jul 14 00:51:21.586975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.586895 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3361b806-60dc-4179-9af7-10d0f2796a1f" containerName="console" Jul 14 00:51:21.586975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.586950 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3361b806-60dc-4179-9af7-10d0f2796a1f" containerName="console" Jul 14 00:51:21.588929 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.588914 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:21.591292 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.591243 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 00:51:21.591465 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.591444 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qtf2m\"" Jul 14 00:51:21.591570 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.591358 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 00:51:21.599887 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.599862 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr"] Jul 14 00:51:21.651880 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.651856 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:21.651993 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.651887 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:21.651993 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.651921 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk78m\" (UniqueName: \"kubernetes.io/projected/c929f4a6-2892-4721-b1a2-3e6fb08703e0-kube-api-access-vk78m\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:21.752872 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.752847 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk78m\" (UniqueName: \"kubernetes.io/projected/c929f4a6-2892-4721-b1a2-3e6fb08703e0-kube-api-access-vk78m\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:21.752988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.752889 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:21.752988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.752920 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:21.753224 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.753207 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:21.753272 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.753246 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:21.760962 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.760946 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk78m\" (UniqueName: \"kubernetes.io/projected/c929f4a6-2892-4721-b1a2-3e6fb08703e0-kube-api-access-vk78m\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:21.902254 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:21.902183 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:22.029386 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:22.029356 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr"] Jul 14 00:51:22.032569 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:51:22.032544 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc929f4a6_2892_4721_b1a2_3e6fb08703e0.slice/crio-cf5ff4f32dadc6dc0c13e688f70856855f0c05f8ff769bfa915d9570c9c50031 WatchSource:0}: Error finding container cf5ff4f32dadc6dc0c13e688f70856855f0c05f8ff769bfa915d9570c9c50031: Status 404 returned error can't find the container with id cf5ff4f32dadc6dc0c13e688f70856855f0c05f8ff769bfa915d9570c9c50031 Jul 14 00:51:22.034440 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:22.034422 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 00:51:22.064928 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:22.064902 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" event={"ID":"c929f4a6-2892-4721-b1a2-3e6fb08703e0","Type":"ContainerStarted","Data":"cf5ff4f32dadc6dc0c13e688f70856855f0c05f8ff769bfa915d9570c9c50031"} Jul 14 00:51:30.096382 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:30.096347 2557 generic.go:356] "Generic (PLEG): container finished" podID="c929f4a6-2892-4721-b1a2-3e6fb08703e0" containerID="5f114a687a98bd802af9a5464852f12e3361e9ab7bf9d4e24b26205ba3e131f9" exitCode=0 Jul 14 00:51:30.096688 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:30.096442 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" event={"ID":"c929f4a6-2892-4721-b1a2-3e6fb08703e0","Type":"ContainerDied","Data":"5f114a687a98bd802af9a5464852f12e3361e9ab7bf9d4e24b26205ba3e131f9"} Jul 14 00:51:32.105900 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:32.105875 2557 generic.go:356] "Generic (PLEG): container finished" podID="c929f4a6-2892-4721-b1a2-3e6fb08703e0" containerID="da62c586f9a6a82a69be1bf002eb7b88d2631cf5ba65d757015fc9b69010985d" exitCode=0 Jul 14 00:51:32.106212 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:32.105934 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" event={"ID":"c929f4a6-2892-4721-b1a2-3e6fb08703e0","Type":"ContainerDied","Data":"da62c586f9a6a82a69be1bf002eb7b88d2631cf5ba65d757015fc9b69010985d"} Jul 14 00:51:41.138814 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:41.138781 2557 generic.go:356] "Generic (PLEG): container finished" podID="c929f4a6-2892-4721-b1a2-3e6fb08703e0" containerID="7a21dfc6e3b2bed503c060032ef1b579807c6b794bcb10495f6b10fab45883c6" exitCode=0 Jul 14 00:51:41.139187 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:41.138827 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" event={"ID":"c929f4a6-2892-4721-b1a2-3e6fb08703e0","Type":"ContainerDied","Data":"7a21dfc6e3b2bed503c060032ef1b579807c6b794bcb10495f6b10fab45883c6"} Jul 14 00:51:42.262263 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:42.262238 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:42.308325 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:42.308297 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-util\") pod \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " Jul 14 00:51:42.308325 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:42.308326 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-bundle\") pod \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " Jul 14 00:51:42.308514 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:42.308369 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vk78m\" (UniqueName: \"kubernetes.io/projected/c929f4a6-2892-4721-b1a2-3e6fb08703e0-kube-api-access-vk78m\") pod \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\" (UID: \"c929f4a6-2892-4721-b1a2-3e6fb08703e0\") " Jul 14 00:51:42.308875 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:42.308851 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-bundle" (OuterVolumeSpecName: "bundle") pod "c929f4a6-2892-4721-b1a2-3e6fb08703e0" (UID: "c929f4a6-2892-4721-b1a2-3e6fb08703e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:51:42.310499 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:42.310475 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c929f4a6-2892-4721-b1a2-3e6fb08703e0-kube-api-access-vk78m" (OuterVolumeSpecName: "kube-api-access-vk78m") pod "c929f4a6-2892-4721-b1a2-3e6fb08703e0" (UID: "c929f4a6-2892-4721-b1a2-3e6fb08703e0"). InnerVolumeSpecName "kube-api-access-vk78m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:51:42.312706 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:42.312681 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-util" (OuterVolumeSpecName: "util") pod "c929f4a6-2892-4721-b1a2-3e6fb08703e0" (UID: "c929f4a6-2892-4721-b1a2-3e6fb08703e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:51:42.409579 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:42.409516 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-util\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:51:42.409579 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:42.409541 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c929f4a6-2892-4721-b1a2-3e6fb08703e0-bundle\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:51:42.409579 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:42.409555 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vk78m\" (UniqueName: \"kubernetes.io/projected/c929f4a6-2892-4721-b1a2-3e6fb08703e0-kube-api-access-vk78m\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:51:43.147564 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:43.147536 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" Jul 14 00:51:43.147720 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:43.147528 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675406lsgr" event={"ID":"c929f4a6-2892-4721-b1a2-3e6fb08703e0","Type":"ContainerDied","Data":"cf5ff4f32dadc6dc0c13e688f70856855f0c05f8ff769bfa915d9570c9c50031"} Jul 14 00:51:43.147720 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:43.147656 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf5ff4f32dadc6dc0c13e688f70856855f0c05f8ff769bfa915d9570c9c50031" Jul 14 00:51:45.233569 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:45.233486 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:46.052391 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:46.052358 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:47.240260 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.240233 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7"] Jul 14 00:51:47.240636 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.240476 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c929f4a6-2892-4721-b1a2-3e6fb08703e0" containerName="pull" Jul 14 00:51:47.240636 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.240487 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c929f4a6-2892-4721-b1a2-3e6fb08703e0" containerName="pull" Jul 14 00:51:47.240636 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.240497 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c929f4a6-2892-4721-b1a2-3e6fb08703e0" containerName="util" Jul 14 00:51:47.240636 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.240503 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c929f4a6-2892-4721-b1a2-3e6fb08703e0" containerName="util" Jul 14 00:51:47.240636 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.240509 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c929f4a6-2892-4721-b1a2-3e6fb08703e0" containerName="extract" Jul 14 00:51:47.240636 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.240515 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c929f4a6-2892-4721-b1a2-3e6fb08703e0" containerName="extract" Jul 14 00:51:47.240636 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.240555 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c929f4a6-2892-4721-b1a2-3e6fb08703e0" containerName="extract" Jul 14 00:51:47.257063 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.257041 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7"] Jul 14 00:51:47.257225 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.257127 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7" Jul 14 00:51:47.259010 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.258989 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 14 00:51:47.259137 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.259022 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 14 00:51:47.259137 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.259095 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lflgn\"" Jul 14 00:51:47.347169 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.347131 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b33124f-113c-434b-b0dc-d8e7e0db79cb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-96wq7\" (UID: \"8b33124f-113c-434b-b0dc-d8e7e0db79cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7" Jul 14 00:51:47.347279 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.347176 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6pj2\" (UniqueName: \"kubernetes.io/projected/8b33124f-113c-434b-b0dc-d8e7e0db79cb-kube-api-access-m6pj2\") pod \"cert-manager-operator-controller-manager-55c444bf86-96wq7\" (UID: \"8b33124f-113c-434b-b0dc-d8e7e0db79cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7" Jul 14 00:51:47.447491 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.447471 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b33124f-113c-434b-b0dc-d8e7e0db79cb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-96wq7\" (UID: \"8b33124f-113c-434b-b0dc-d8e7e0db79cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7" Jul 14 00:51:47.447611 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.447497 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6pj2\" (UniqueName: \"kubernetes.io/projected/8b33124f-113c-434b-b0dc-d8e7e0db79cb-kube-api-access-m6pj2\") pod \"cert-manager-operator-controller-manager-55c444bf86-96wq7\" (UID: \"8b33124f-113c-434b-b0dc-d8e7e0db79cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7" Jul 14 00:51:47.447829 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.447809 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8b33124f-113c-434b-b0dc-d8e7e0db79cb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-96wq7\" (UID: \"8b33124f-113c-434b-b0dc-d8e7e0db79cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7" Jul 14 00:51:47.454387 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.454369 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6pj2\" (UniqueName: \"kubernetes.io/projected/8b33124f-113c-434b-b0dc-d8e7e0db79cb-kube-api-access-m6pj2\") pod \"cert-manager-operator-controller-manager-55c444bf86-96wq7\" (UID: \"8b33124f-113c-434b-b0dc-d8e7e0db79cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7" Jul 14 00:51:47.567449 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.567421 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7" Jul 14 00:51:47.711826 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:47.711797 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7"] Jul 14 00:51:47.715433 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:51:47.715408 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b33124f_113c_434b_b0dc_d8e7e0db79cb.slice/crio-76b5c445d539d5a8aa2fb23ef52b6115bd3fecf347e6bce09238849d9212960e WatchSource:0}: Error finding container 76b5c445d539d5a8aa2fb23ef52b6115bd3fecf347e6bce09238849d9212960e: Status 404 returned error can't find the container with id 76b5c445d539d5a8aa2fb23ef52b6115bd3fecf347e6bce09238849d9212960e Jul 14 00:51:48.166785 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:48.166750 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7" event={"ID":"8b33124f-113c-434b-b0dc-d8e7e0db79cb","Type":"ContainerStarted","Data":"76b5c445d539d5a8aa2fb23ef52b6115bd3fecf347e6bce09238849d9212960e"} Jul 14 00:51:48.850554 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:48.850505 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:48.959084 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:48.959052 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:50.177000 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:50.176960 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7" event={"ID":"8b33124f-113c-434b-b0dc-d8e7e0db79cb","Type":"ContainerStarted","Data":"113b2633cdc04c7a95a8d5fcb895587895166a275f0b72b41a2a519cd6697b5f"} Jul 14 00:51:50.196399 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:50.196347 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-96wq7" podStartSLOduration=1.482955296 podStartE2EDuration="3.196327289s" totalImagesPullingTime="1.713371993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:51:50.194280555 +0000 UTC m=+447.045647924" watchObservedRunningTime="2026-07-14 00:51:50.196327289 +0000 UTC m=+447.047694662" Jul 14 00:51:50.581413 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:50.581385 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:50.690025 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:50.689991 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:51.190451 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:51.190416 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:51.297411 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:51.297375 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:53.360107 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:53.360071 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:53.749977 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:53.749891 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:55.844338 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:55.844307 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:51:55.944697 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:51:55.944667 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:01.367757 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:01.367710 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:03.597557 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.597524 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sp4x5"] Jul 14 00:52:03.599614 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.599598 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sp4x5" Jul 14 00:52:03.601431 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.601403 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 00:52:03.601547 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.601446 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 00:52:03.601547 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.601451 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-w46m7\"" Jul 14 00:52:03.610287 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.610268 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sp4x5"] Jul 14 00:52:03.663503 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.663478 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tpx4\" (UniqueName: \"kubernetes.io/projected/4bf78a6b-97b4-444d-bbe4-479bdc905874-kube-api-access-9tpx4\") pod \"cert-manager-55b4d954d5-sp4x5\" (UID: \"4bf78a6b-97b4-444d-bbe4-479bdc905874\") " pod="cert-manager/cert-manager-55b4d954d5-sp4x5" Jul 14 00:52:03.663631 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.663521 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bf78a6b-97b4-444d-bbe4-479bdc905874-bound-sa-token\") pod \"cert-manager-55b4d954d5-sp4x5\" (UID: \"4bf78a6b-97b4-444d-bbe4-479bdc905874\") " pod="cert-manager/cert-manager-55b4d954d5-sp4x5" Jul 14 00:52:03.764779 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.764758 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bf78a6b-97b4-444d-bbe4-479bdc905874-bound-sa-token\") pod \"cert-manager-55b4d954d5-sp4x5\" (UID: \"4bf78a6b-97b4-444d-bbe4-479bdc905874\") " pod="cert-manager/cert-manager-55b4d954d5-sp4x5" Jul 14 00:52:03.764902 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.764831 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tpx4\" (UniqueName: \"kubernetes.io/projected/4bf78a6b-97b4-444d-bbe4-479bdc905874-kube-api-access-9tpx4\") pod \"cert-manager-55b4d954d5-sp4x5\" (UID: \"4bf78a6b-97b4-444d-bbe4-479bdc905874\") " pod="cert-manager/cert-manager-55b4d954d5-sp4x5" Jul 14 00:52:03.771784 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.771761 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bf78a6b-97b4-444d-bbe4-479bdc905874-bound-sa-token\") pod \"cert-manager-55b4d954d5-sp4x5\" (UID: \"4bf78a6b-97b4-444d-bbe4-479bdc905874\") " pod="cert-manager/cert-manager-55b4d954d5-sp4x5" Jul 14 00:52:03.771876 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.771770 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tpx4\" (UniqueName: \"kubernetes.io/projected/4bf78a6b-97b4-444d-bbe4-479bdc905874-kube-api-access-9tpx4\") pod \"cert-manager-55b4d954d5-sp4x5\" (UID: \"4bf78a6b-97b4-444d-bbe4-479bdc905874\") " pod="cert-manager/cert-manager-55b4d954d5-sp4x5" Jul 14 00:52:03.851857 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.851794 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x8tvn"] Jul 14 00:52:03.853915 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.853897 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" Jul 14 00:52:03.855534 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.855515 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-52pwp\"" Jul 14 00:52:03.866127 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.866106 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x8tvn"] Jul 14 00:52:03.908216 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.908194 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sp4x5" Jul 14 00:52:03.966814 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.966747 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edf63e2c-e0b2-4f58-afe2-044723f20ec1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x8tvn\" (UID: \"edf63e2c-e0b2-4f58-afe2-044723f20ec1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" Jul 14 00:52:03.967260 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:03.967141 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sjvt\" (UniqueName: \"kubernetes.io/projected/edf63e2c-e0b2-4f58-afe2-044723f20ec1-kube-api-access-9sjvt\") pod \"cert-manager-webhook-6486ffd997-x8tvn\" (UID: \"edf63e2c-e0b2-4f58-afe2-044723f20ec1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" Jul 14 00:52:04.029712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:04.029502 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sp4x5"] Jul 14 00:52:04.033168 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:52:04.033092 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bf78a6b_97b4_444d_bbe4_479bdc905874.slice/crio-84ffc5681d87ef6b5957c335275cdab645529b9cff1ae92b094821513bd102f6 WatchSource:0}: Error finding container 84ffc5681d87ef6b5957c335275cdab645529b9cff1ae92b094821513bd102f6: Status 404 returned error can't find the container with id 84ffc5681d87ef6b5957c335275cdab645529b9cff1ae92b094821513bd102f6 Jul 14 00:52:04.068436 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:04.068414 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edf63e2c-e0b2-4f58-afe2-044723f20ec1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x8tvn\" (UID: \"edf63e2c-e0b2-4f58-afe2-044723f20ec1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" Jul 14 00:52:04.068546 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:04.068464 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sjvt\" (UniqueName: \"kubernetes.io/projected/edf63e2c-e0b2-4f58-afe2-044723f20ec1-kube-api-access-9sjvt\") pod \"cert-manager-webhook-6486ffd997-x8tvn\" (UID: \"edf63e2c-e0b2-4f58-afe2-044723f20ec1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" Jul 14 00:52:04.076396 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:04.076369 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sjvt\" (UniqueName: \"kubernetes.io/projected/edf63e2c-e0b2-4f58-afe2-044723f20ec1-kube-api-access-9sjvt\") pod \"cert-manager-webhook-6486ffd997-x8tvn\" (UID: \"edf63e2c-e0b2-4f58-afe2-044723f20ec1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" Jul 14 00:52:04.076476 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:04.076460 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/edf63e2c-e0b2-4f58-afe2-044723f20ec1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-x8tvn\" (UID: \"edf63e2c-e0b2-4f58-afe2-044723f20ec1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" Jul 14 00:52:04.162853 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:04.162789 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" Jul 14 00:52:04.229406 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:04.229375 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sp4x5" event={"ID":"4bf78a6b-97b4-444d-bbe4-479bdc905874","Type":"ContainerStarted","Data":"84ffc5681d87ef6b5957c335275cdab645529b9cff1ae92b094821513bd102f6"} Jul 14 00:52:04.284535 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:04.284502 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-x8tvn"] Jul 14 00:52:04.287464 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:52:04.287442 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podedf63e2c_e0b2_4f58_afe2_044723f20ec1.slice/crio-83a4d4760cc16e317beabfe8603a3cf15b0da0fda7e515e4b4370ed6b052a419 WatchSource:0}: Error finding container 83a4d4760cc16e317beabfe8603a3cf15b0da0fda7e515e4b4370ed6b052a419: Status 404 returned error can't find the container with id 83a4d4760cc16e317beabfe8603a3cf15b0da0fda7e515e4b4370ed6b052a419 Jul 14 00:52:05.235928 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:05.235863 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" event={"ID":"edf63e2c-e0b2-4f58-afe2-044723f20ec1","Type":"ContainerStarted","Data":"83a4d4760cc16e317beabfe8603a3cf15b0da0fda7e515e4b4370ed6b052a419"} Jul 14 00:52:06.904124 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:06.904092 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:06.959673 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:06.959603 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:07.245462 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:07.245390 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sp4x5" event={"ID":"4bf78a6b-97b4-444d-bbe4-479bdc905874","Type":"ContainerStarted","Data":"54a8bc83a516cfe26299a66ac57b1828efe947cad09fdcdd643d25daa30805ed"} Jul 14 00:52:07.246790 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:07.246766 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" event={"ID":"edf63e2c-e0b2-4f58-afe2-044723f20ec1","Type":"ContainerStarted","Data":"9e1dcdfd9d02f26b46e855dbdb372a913f38c19be1c688723ac6c1f8d25ecfd4"} Jul 14 00:52:07.246925 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:07.246817 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" Jul 14 00:52:07.260034 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:07.259984 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-sp4x5" podStartSLOduration=1.6239751409999998 podStartE2EDuration="4.259973541s" totalImagesPullingTime="2.6359984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:52:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:52:07.258595348 +0000 UTC m=+464.109962722" watchObservedRunningTime="2026-07-14 00:52:07.259973541 +0000 UTC m=+464.111340910" Jul 14 00:52:07.271589 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:07.271528 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" podStartSLOduration=1.897972707 podStartE2EDuration="4.271516567s" totalImagesPullingTime="2.37354386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:52:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:52:07.271186674 +0000 UTC m=+464.122554043" watchObservedRunningTime="2026-07-14 00:52:07.271516567 +0000 UTC m=+464.122883942" Jul 14 00:52:13.253836 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:13.253806 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-x8tvn" Jul 14 00:52:19.044716 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:19.044676 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:19.143697 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:19.143662 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:24.399679 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:24.399637 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:24.500018 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:24.499979 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:26.287507 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:26.287475 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:29.126166 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.126126 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm"] Jul 14 00:52:29.128557 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.128541 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:29.130221 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.130202 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qtf2m\"" Jul 14 00:52:29.130308 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.130203 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 14 00:52:29.130308 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.130242 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 14 00:52:29.136422 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.136397 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm"] Jul 14 00:52:29.246080 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.246054 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:29.246210 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.246084 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkct7\" (UniqueName: \"kubernetes.io/projected/83584cd7-57ee-4edf-8e94-78508842643f-kube-api-access-tkct7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:29.246210 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.246115 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:29.346394 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.346366 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:29.346523 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.346416 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:29.346523 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.346435 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkct7\" (UniqueName: \"kubernetes.io/projected/83584cd7-57ee-4edf-8e94-78508842643f-kube-api-access-tkct7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:29.346754 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.346736 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:29.346798 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.346774 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:29.354650 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.354616 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkct7\" (UniqueName: \"kubernetes.io/projected/83584cd7-57ee-4edf-8e94-78508842643f-kube-api-access-tkct7\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:29.437654 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.437605 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:29.560579 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:29.560476 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm"] Jul 14 00:52:29.563222 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:52:29.563194 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83584cd7_57ee_4edf_8e94_78508842643f.slice/crio-624c20b04e5be98b01238a100579015ad8e9e7838c1851d03afc401dd0448d70 WatchSource:0}: Error finding container 624c20b04e5be98b01238a100579015ad8e9e7838c1851d03afc401dd0448d70: Status 404 returned error can't find the container with id 624c20b04e5be98b01238a100579015ad8e9e7838c1851d03afc401dd0448d70 Jul 14 00:52:30.344366 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:30.344331 2557 generic.go:356] "Generic (PLEG): container finished" podID="83584cd7-57ee-4edf-8e94-78508842643f" containerID="c6591acc1a59a93f005beadf7ed2111d8a3f73c371149b11343dbd036a7f07a9" exitCode=0 Jul 14 00:52:30.344739 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:30.344404 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" event={"ID":"83584cd7-57ee-4edf-8e94-78508842643f","Type":"ContainerDied","Data":"c6591acc1a59a93f005beadf7ed2111d8a3f73c371149b11343dbd036a7f07a9"} Jul 14 00:52:30.344739 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:30.344461 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" event={"ID":"83584cd7-57ee-4edf-8e94-78508842643f","Type":"ContainerStarted","Data":"624c20b04e5be98b01238a100579015ad8e9e7838c1851d03afc401dd0448d70"} Jul 14 00:52:31.349721 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:31.349634 2557 generic.go:356] "Generic (PLEG): container finished" podID="83584cd7-57ee-4edf-8e94-78508842643f" containerID="edf6624d657e8e3385c5f4de1d4b5ab60cacb3ad72c9fb132d18edb25cc3bf1a" exitCode=0 Jul 14 00:52:31.349721 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:31.349683 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" event={"ID":"83584cd7-57ee-4edf-8e94-78508842643f","Type":"ContainerDied","Data":"edf6624d657e8e3385c5f4de1d4b5ab60cacb3ad72c9fb132d18edb25cc3bf1a"} Jul 14 00:52:32.355666 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:32.355629 2557 generic.go:356] "Generic (PLEG): container finished" podID="83584cd7-57ee-4edf-8e94-78508842643f" containerID="52ceccf999c4d1ac853282a6448b078885b5f4e32305c8d8ca9ccb15669012fd" exitCode=0 Jul 14 00:52:32.356051 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:32.355716 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" event={"ID":"83584cd7-57ee-4edf-8e94-78508842643f","Type":"ContainerDied","Data":"52ceccf999c4d1ac853282a6448b078885b5f4e32305c8d8ca9ccb15669012fd"} Jul 14 00:52:33.478141 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:33.478119 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:33.576325 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:33.576302 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkct7\" (UniqueName: \"kubernetes.io/projected/83584cd7-57ee-4edf-8e94-78508842643f-kube-api-access-tkct7\") pod \"83584cd7-57ee-4edf-8e94-78508842643f\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " Jul 14 00:52:33.576459 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:33.576331 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-util\") pod \"83584cd7-57ee-4edf-8e94-78508842643f\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " Jul 14 00:52:33.576459 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:33.576402 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-bundle\") pod \"83584cd7-57ee-4edf-8e94-78508842643f\" (UID: \"83584cd7-57ee-4edf-8e94-78508842643f\") " Jul 14 00:52:33.577122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:33.577098 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-bundle" (OuterVolumeSpecName: "bundle") pod "83584cd7-57ee-4edf-8e94-78508842643f" (UID: "83584cd7-57ee-4edf-8e94-78508842643f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:52:33.578295 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:33.578274 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83584cd7-57ee-4edf-8e94-78508842643f-kube-api-access-tkct7" (OuterVolumeSpecName: "kube-api-access-tkct7") pod "83584cd7-57ee-4edf-8e94-78508842643f" (UID: "83584cd7-57ee-4edf-8e94-78508842643f"). InnerVolumeSpecName "kube-api-access-tkct7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:52:33.582381 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:33.582357 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-util" (OuterVolumeSpecName: "util") pod "83584cd7-57ee-4edf-8e94-78508842643f" (UID: "83584cd7-57ee-4edf-8e94-78508842643f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:52:33.676866 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:33.676809 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkct7\" (UniqueName: \"kubernetes.io/projected/83584cd7-57ee-4edf-8e94-78508842643f-kube-api-access-tkct7\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:52:33.676866 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:33.676827 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-util\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:52:33.676866 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:33.676837 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/83584cd7-57ee-4edf-8e94-78508842643f-bundle\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:52:34.365442 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:34.365410 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" Jul 14 00:52:34.365598 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:34.365409 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affzkqm" event={"ID":"83584cd7-57ee-4edf-8e94-78508842643f","Type":"ContainerDied","Data":"624c20b04e5be98b01238a100579015ad8e9e7838c1851d03afc401dd0448d70"} Jul 14 00:52:34.365598 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:34.365531 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="624c20b04e5be98b01238a100579015ad8e9e7838c1851d03afc401dd0448d70" Jul 14 00:52:40.689286 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:40.689251 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:40.791070 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:40.791039 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:40.963313 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:40.963231 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:41.058548 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.058518 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:41.495389 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.495359 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:41.688103 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.688076 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:41.792954 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.792916 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:41.818487 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.818463 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd"] Jul 14 00:52:41.818735 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.818722 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83584cd7-57ee-4edf-8e94-78508842643f" containerName="extract" Jul 14 00:52:41.818779 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.818737 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="83584cd7-57ee-4edf-8e94-78508842643f" containerName="extract" Jul 14 00:52:41.818779 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.818756 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83584cd7-57ee-4edf-8e94-78508842643f" containerName="util" Jul 14 00:52:41.818779 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.818761 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="83584cd7-57ee-4edf-8e94-78508842643f" containerName="util" Jul 14 00:52:41.818779 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.818769 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83584cd7-57ee-4edf-8e94-78508842643f" containerName="pull" Jul 14 00:52:41.818779 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.818777 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="83584cd7-57ee-4edf-8e94-78508842643f" containerName="pull" Jul 14 00:52:41.818932 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.818825 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="83584cd7-57ee-4edf-8e94-78508842643f" containerName="extract" Jul 14 00:52:41.823141 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.823122 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.826427 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:52:41.826400 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-service-cert\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" type="*v1.Secret" Jul 14 00:52:41.826523 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:52:41.826400 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-manager-dockercfg-k2lvq\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k2lvq\"" type="*v1.Secret" Jul 14 00:52:41.826572 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:52:41.826551 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 14 00:52:41.826607 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:52:41.826571 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"opendatahub-operator-controller-webhook-cert\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" type="*v1.Secret" Jul 14 00:52:41.827446 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:52:41.827428 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 14 00:52:41.827499 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:52:41.827449 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="0a2ecbb3-980b-4ad5-a899-a92616131a09" pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.835445 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.835422 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a2ecbb3-980b-4ad5-a899-a92616131a09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.835546 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.835462 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a2ecbb3-980b-4ad5-a899-a92616131a09-tmp\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.835611 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.835555 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a2ecbb3-980b-4ad5-a899-a92616131a09-manifests\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.835611 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.835594 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbxrc\" (UniqueName: \"kubernetes.io/projected/0a2ecbb3-980b-4ad5-a899-a92616131a09-kube-api-access-sbxrc\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.835699 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.835649 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a2ecbb3-980b-4ad5-a899-a92616131a09-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.860715 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.860690 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd"] Jul 14 00:52:41.936568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.936543 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a2ecbb3-980b-4ad5-a899-a92616131a09-manifests\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.936705 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.936576 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbxrc\" (UniqueName: \"kubernetes.io/projected/0a2ecbb3-980b-4ad5-a899-a92616131a09-kube-api-access-sbxrc\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.936705 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.936609 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a2ecbb3-980b-4ad5-a899-a92616131a09-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.936796 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.936746 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a2ecbb3-980b-4ad5-a899-a92616131a09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.936845 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.936820 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a2ecbb3-980b-4ad5-a899-a92616131a09-tmp\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.937013 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.936993 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0a2ecbb3-980b-4ad5-a899-a92616131a09-manifests\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:41.939011 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:41.938991 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0a2ecbb3-980b-4ad5-a899-a92616131a09-tmp\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:42.019543 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:42.019515 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:42.120058 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:42.119996 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:42.219725 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:42.219699 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:42.319285 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:42.319249 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:42.390891 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:42.390818 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:42.501242 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:42.501208 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:42.585536 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:42.585506 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:52:42.891882 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:42.891858 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 00:52:42.899703 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:42.899676 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0a2ecbb3-980b-4ad5-a899-a92616131a09-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:42.899904 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:42.899878 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0a2ecbb3-980b-4ad5-a899-a92616131a09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:42.950182 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:52:42.950144 2557 projected.go:291] Couldn't get configMap opendatahub/kube-root-ca.crt: failed to sync configmap cache: timed out waiting for the condition Jul 14 00:52:42.986189 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:42.986132 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 00:52:43.234013 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:43.233950 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 00:52:43.240411 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:52:43.240393 2557 projected.go:196] Error preparing data for projected volume kube-api-access-sbxrc for pod opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd: failed to sync configmap cache: timed out waiting for the condition Jul 14 00:52:43.240504 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:52:43.240461 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0a2ecbb3-980b-4ad5-a899-a92616131a09-kube-api-access-sbxrc podName:0a2ecbb3-980b-4ad5-a899-a92616131a09 nodeName:}" failed. No retries permitted until 2026-07-14 00:52:43.740444966 +0000 UTC m=+500.591812315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sbxrc" (UniqueName: "kubernetes.io/projected/0a2ecbb3-980b-4ad5-a899-a92616131a09-kube-api-access-sbxrc") pod "opendatahub-operator-controller-manager-6b7d6985f7-25whd" (UID: "0a2ecbb3-980b-4ad5-a899-a92616131a09") : failed to sync configmap cache: timed out waiting for the condition Jul 14 00:52:43.251193 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:43.251171 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k2lvq\"" Jul 14 00:52:43.264787 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:43.264766 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 00:52:43.748013 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:43.747985 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbxrc\" (UniqueName: \"kubernetes.io/projected/0a2ecbb3-980b-4ad5-a899-a92616131a09-kube-api-access-sbxrc\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:43.750450 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:43.750421 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbxrc\" (UniqueName: \"kubernetes.io/projected/0a2ecbb3-980b-4ad5-a899-a92616131a09-kube-api-access-sbxrc\") pod \"opendatahub-operator-controller-manager-6b7d6985f7-25whd\" (UID: \"0a2ecbb3-980b-4ad5-a899-a92616131a09\") " pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:43.933212 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:43.933178 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:44.063235 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:44.063205 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd"] Jul 14 00:52:44.065871 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:52:44.065845 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a2ecbb3_980b_4ad5_a899_a92616131a09.slice/crio-48c729ccb811a31304aed5b99d057340d0fccbd07466b98222e841f6d84d8fd7 WatchSource:0}: Error finding container 48c729ccb811a31304aed5b99d057340d0fccbd07466b98222e841f6d84d8fd7: Status 404 returned error can't find the container with id 48c729ccb811a31304aed5b99d057340d0fccbd07466b98222e841f6d84d8fd7 Jul 14 00:52:44.408076 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:44.408039 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" event={"ID":"0a2ecbb3-980b-4ad5-a899-a92616131a09","Type":"ContainerStarted","Data":"48c729ccb811a31304aed5b99d057340d0fccbd07466b98222e841f6d84d8fd7"} Jul 14 00:52:47.426915 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:47.426878 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" event={"ID":"0a2ecbb3-980b-4ad5-a899-a92616131a09","Type":"ContainerStarted","Data":"0b7f874819f0d6b96fac679d0d33f741b04a92b49e89f5f813f0d3f91efbafb0"} Jul 14 00:52:48.432199 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:48.432139 2557 generic.go:356] "Generic (PLEG): container finished" podID="0a2ecbb3-980b-4ad5-a899-a92616131a09" containerID="0b7f874819f0d6b96fac679d0d33f741b04a92b49e89f5f813f0d3f91efbafb0" exitCode=0 Jul 14 00:52:48.432199 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:48.432181 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" event={"ID":"0a2ecbb3-980b-4ad5-a899-a92616131a09","Type":"ContainerDied","Data":"0b7f874819f0d6b96fac679d0d33f741b04a92b49e89f5f813f0d3f91efbafb0"} Jul 14 00:52:49.444436 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:49.444395 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" event={"ID":"0a2ecbb3-980b-4ad5-a899-a92616131a09","Type":"ContainerStarted","Data":"dd2ef2188c57fe247e137cf518cf1bcf278760da18ef90f599e0408f9b30fb5f"} Jul 14 00:52:49.444830 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:49.444523 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:52:49.464694 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:52:49.464638 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" podStartSLOduration=4.342823931 podStartE2EDuration="8.464621486s" totalImagesPullingTime="3.092376354s" totalInitContainerRuntime="1.029421201s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:52:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:52:49.464340278 +0000 UTC m=+506.315707648" watchObservedRunningTime="2026-07-14 00:52:49.464621486 +0000 UTC m=+506.315988859" Jul 14 00:53:00.450640 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:00.450595 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b7d6985f7-25whd" Jul 14 00:53:02.019560 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.019528 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:02.141321 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.141286 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:02.656899 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.656862 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:02.666486 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.666458 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r"] Jul 14 00:53:02.674888 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.674859 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" Jul 14 00:53:02.676998 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.676976 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 14 00:53:02.677105 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.676999 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 14 00:53:02.677281 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.677267 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 00:53:02.677435 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.677416 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 00:53:02.680672 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.680616 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r"] Jul 14 00:53:02.783485 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.783449 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9v56\" (UniqueName: \"kubernetes.io/projected/a656bf6b-2f6c-4e6d-aee0-998cb9aafc22-kube-api-access-m9v56\") pod \"kube-auth-proxy-5f876ccc94-tgk5r\" (UID: \"a656bf6b-2f6c-4e6d-aee0-998cb9aafc22\") " pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" Jul 14 00:53:02.783606 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.783503 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a656bf6b-2f6c-4e6d-aee0-998cb9aafc22-tls-certs\") pod \"kube-auth-proxy-5f876ccc94-tgk5r\" (UID: \"a656bf6b-2f6c-4e6d-aee0-998cb9aafc22\") " pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" Jul 14 00:53:02.783606 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.783537 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a656bf6b-2f6c-4e6d-aee0-998cb9aafc22-tmp\") pod \"kube-auth-proxy-5f876ccc94-tgk5r\" (UID: \"a656bf6b-2f6c-4e6d-aee0-998cb9aafc22\") " pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" Jul 14 00:53:02.884654 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.884625 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9v56\" (UniqueName: \"kubernetes.io/projected/a656bf6b-2f6c-4e6d-aee0-998cb9aafc22-kube-api-access-m9v56\") pod \"kube-auth-proxy-5f876ccc94-tgk5r\" (UID: \"a656bf6b-2f6c-4e6d-aee0-998cb9aafc22\") " pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" Jul 14 00:53:02.884802 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.884669 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a656bf6b-2f6c-4e6d-aee0-998cb9aafc22-tls-certs\") pod \"kube-auth-proxy-5f876ccc94-tgk5r\" (UID: \"a656bf6b-2f6c-4e6d-aee0-998cb9aafc22\") " pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" Jul 14 00:53:02.884802 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.884695 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a656bf6b-2f6c-4e6d-aee0-998cb9aafc22-tmp\") pod \"kube-auth-proxy-5f876ccc94-tgk5r\" (UID: \"a656bf6b-2f6c-4e6d-aee0-998cb9aafc22\") " pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" Jul 14 00:53:02.886940 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.886915 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a656bf6b-2f6c-4e6d-aee0-998cb9aafc22-tmp\") pod \"kube-auth-proxy-5f876ccc94-tgk5r\" (UID: \"a656bf6b-2f6c-4e6d-aee0-998cb9aafc22\") " pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" Jul 14 00:53:02.887293 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.887270 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a656bf6b-2f6c-4e6d-aee0-998cb9aafc22-tls-certs\") pod \"kube-auth-proxy-5f876ccc94-tgk5r\" (UID: \"a656bf6b-2f6c-4e6d-aee0-998cb9aafc22\") " pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" Jul 14 00:53:02.900641 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.900612 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kjxkv"] Jul 14 00:53:02.904087 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.904062 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9v56\" (UniqueName: \"kubernetes.io/projected/a656bf6b-2f6c-4e6d-aee0-998cb9aafc22-kube-api-access-m9v56\") pod \"kube-auth-proxy-5f876ccc94-tgk5r\" (UID: \"a656bf6b-2f6c-4e6d-aee0-998cb9aafc22\") " pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" Jul 14 00:53:02.904537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.904520 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" Jul 14 00:53:02.906800 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.906779 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-68rd9\"" Jul 14 00:53:02.907004 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.906966 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 14 00:53:02.916593 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.916569 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kjxkv"] Jul 14 00:53:02.934645 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.934612 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:02.985635 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.985611 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" Jul 14 00:53:02.985792 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.985771 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18bbc1d0-0722-40c4-b6aa-c400cc1882ed-cert\") pod \"odh-model-controller-7998c67dc4-kjxkv\" (UID: \"18bbc1d0-0722-40c4-b6aa-c400cc1882ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" Jul 14 00:53:02.985849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.985821 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7tdx\" (UniqueName: \"kubernetes.io/projected/18bbc1d0-0722-40c4-b6aa-c400cc1882ed-kube-api-access-g7tdx\") pod \"odh-model-controller-7998c67dc4-kjxkv\" (UID: \"18bbc1d0-0722-40c4-b6aa-c400cc1882ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" Jul 14 00:53:02.993794 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:02.993770 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:03.086991 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.086956 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18bbc1d0-0722-40c4-b6aa-c400cc1882ed-cert\") pod \"odh-model-controller-7998c67dc4-kjxkv\" (UID: \"18bbc1d0-0722-40c4-b6aa-c400cc1882ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" Jul 14 00:53:03.087419 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.087051 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7tdx\" (UniqueName: \"kubernetes.io/projected/18bbc1d0-0722-40c4-b6aa-c400cc1882ed-kube-api-access-g7tdx\") pod \"odh-model-controller-7998c67dc4-kjxkv\" (UID: \"18bbc1d0-0722-40c4-b6aa-c400cc1882ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" Jul 14 00:53:03.087419 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:53:03.087121 2557 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 14 00:53:03.087419 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:53:03.087205 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18bbc1d0-0722-40c4-b6aa-c400cc1882ed-cert podName:18bbc1d0-0722-40c4-b6aa-c400cc1882ed nodeName:}" failed. No retries permitted until 2026-07-14 00:53:03.587182755 +0000 UTC m=+520.438550102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/18bbc1d0-0722-40c4-b6aa-c400cc1882ed-cert") pod "odh-model-controller-7998c67dc4-kjxkv" (UID: "18bbc1d0-0722-40c4-b6aa-c400cc1882ed") : secret "odh-model-controller-webhook-cert" not found Jul 14 00:53:03.102468 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.102443 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7tdx\" (UniqueName: \"kubernetes.io/projected/18bbc1d0-0722-40c4-b6aa-c400cc1882ed-kube-api-access-g7tdx\") pod \"odh-model-controller-7998c67dc4-kjxkv\" (UID: \"18bbc1d0-0722-40c4-b6aa-c400cc1882ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" Jul 14 00:53:03.122424 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.122389 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:03.123695 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.123669 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r"] Jul 14 00:53:03.126858 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:53:03.126834 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda656bf6b_2f6c_4e6d_aee0_998cb9aafc22.slice/crio-398b1bbb55cbfd6eb4c2cd6cef00b254f88404dace42c1a490d97e3274900cae WatchSource:0}: Error finding container 398b1bbb55cbfd6eb4c2cd6cef00b254f88404dace42c1a490d97e3274900cae: Status 404 returned error can't find the container with id 398b1bbb55cbfd6eb4c2cd6cef00b254f88404dace42c1a490d97e3274900cae Jul 14 00:53:03.219043 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.218984 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:03.420118 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.420090 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:03.512038 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.511961 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" event={"ID":"a656bf6b-2f6c-4e6d-aee0-998cb9aafc22","Type":"ContainerStarted","Data":"398b1bbb55cbfd6eb4c2cd6cef00b254f88404dace42c1a490d97e3274900cae"} Jul 14 00:53:03.591104 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.591074 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18bbc1d0-0722-40c4-b6aa-c400cc1882ed-cert\") pod \"odh-model-controller-7998c67dc4-kjxkv\" (UID: \"18bbc1d0-0722-40c4-b6aa-c400cc1882ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" Jul 14 00:53:03.593843 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.593824 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/18bbc1d0-0722-40c4-b6aa-c400cc1882ed-cert\") pod \"odh-model-controller-7998c67dc4-kjxkv\" (UID: \"18bbc1d0-0722-40c4-b6aa-c400cc1882ed\") " pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" Jul 14 00:53:03.823961 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.823927 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" Jul 14 00:53:03.830643 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.828471 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c8skl"] Jul 14 00:53:03.832060 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.832042 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" Jul 14 00:53:03.834017 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.833987 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 14 00:53:03.846738 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.846715 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c8skl"] Jul 14 00:53:03.865783 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.865749 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:03.895224 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.895007 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggkmq\" (UniqueName: \"kubernetes.io/projected/ea299afa-3425-445e-817f-469542f41680-kube-api-access-ggkmq\") pod \"model-serving-api-54d89bd5fc-c8skl\" (UID: \"ea299afa-3425-445e-817f-469542f41680\") " pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" Jul 14 00:53:03.895224 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.895097 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea299afa-3425-445e-817f-469542f41680-tls-certs\") pod \"model-serving-api-54d89bd5fc-c8skl\" (UID: \"ea299afa-3425-445e-817f-469542f41680\") " pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" Jul 14 00:53:03.996585 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.996559 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggkmq\" (UniqueName: \"kubernetes.io/projected/ea299afa-3425-445e-817f-469542f41680-kube-api-access-ggkmq\") pod \"model-serving-api-54d89bd5fc-c8skl\" (UID: \"ea299afa-3425-445e-817f-469542f41680\") " pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" Jul 14 00:53:03.996658 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.996621 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-kjxkv"] Jul 14 00:53:03.996711 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.996657 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea299afa-3425-445e-817f-469542f41680-tls-certs\") pod \"model-serving-api-54d89bd5fc-c8skl\" (UID: \"ea299afa-3425-445e-817f-469542f41680\") " pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" Jul 14 00:53:03.999019 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:53:03.998978 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18bbc1d0_0722_40c4_b6aa_c400cc1882ed.slice/crio-54b825157f2cab5bc842b7aed5749fac9d9293daa9eb5aef24b93c688a88cbc8 WatchSource:0}: Error finding container 54b825157f2cab5bc842b7aed5749fac9d9293daa9eb5aef24b93c688a88cbc8: Status 404 returned error can't find the container with id 54b825157f2cab5bc842b7aed5749fac9d9293daa9eb5aef24b93c688a88cbc8 Jul 14 00:53:03.999576 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:03.999555 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea299afa-3425-445e-817f-469542f41680-tls-certs\") pod \"model-serving-api-54d89bd5fc-c8skl\" (UID: \"ea299afa-3425-445e-817f-469542f41680\") " pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" Jul 14 00:53:04.020021 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.019943 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggkmq\" (UniqueName: \"kubernetes.io/projected/ea299afa-3425-445e-817f-469542f41680-kube-api-access-ggkmq\") pod \"model-serving-api-54d89bd5fc-c8skl\" (UID: \"ea299afa-3425-445e-817f-469542f41680\") " pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" Jul 14 00:53:04.048462 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.048436 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:04.143059 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.142979 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" Jul 14 00:53:04.287550 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.287521 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c8skl"] Jul 14 00:53:04.290002 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:53:04.289976 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea299afa_3425_445e_817f_469542f41680.slice/crio-c90a9c486048b43c1b0cca846441a048cadf1c63e5ad5521a7bd88432d707d41 WatchSource:0}: Error finding container c90a9c486048b43c1b0cca846441a048cadf1c63e5ad5521a7bd88432d707d41: Status 404 returned error can't find the container with id c90a9c486048b43c1b0cca846441a048cadf1c63e5ad5521a7bd88432d707d41 Jul 14 00:53:04.367089 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.367064 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:04.367245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.367225 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5"] Jul 14 00:53:04.371466 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.371451 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.373309 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.373288 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 00:53:04.373402 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.373329 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 00:53:04.373402 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.373342 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-h5472\"" Jul 14 00:53:04.373402 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.373295 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 00:53:04.373589 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.373575 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 00:53:04.383240 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.383220 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5"] Jul 14 00:53:04.399571 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.399516 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.399571 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.399547 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-cacerts\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.399571 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.399565 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.399712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.399583 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8732d060-6b69-4aed-b89a-e836f3ad3a3a-local-certs\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.399712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.399687 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.399778 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.399711 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmt7k\" (UniqueName: \"kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-kube-api-access-fmt7k\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.399778 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.399740 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-token\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.434511 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.434485 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:04.500491 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.500460 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.500663 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.500512 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-cacerts\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.500663 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.500539 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.500663 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.500563 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8732d060-6b69-4aed-b89a-e836f3ad3a3a-local-certs\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.500663 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.500627 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.500663 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.500651 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmt7k\" (UniqueName: \"kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-kube-api-access-fmt7k\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.500926 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.500685 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-token\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.501692 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.501640 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.503439 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.503416 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8732d060-6b69-4aed-b89a-e836f3ad3a3a-local-certs\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.503535 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.503487 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.503535 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.503495 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-cacerts\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.503665 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.503553 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.508741 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.508722 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-token\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.509145 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.509124 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmt7k\" (UniqueName: \"kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-kube-api-access-fmt7k\") pod \"istiod-openshift-gateway-7574459656-5fhp5\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.519234 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.519197 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" event={"ID":"ea299afa-3425-445e-817f-469542f41680","Type":"ContainerStarted","Data":"c90a9c486048b43c1b0cca846441a048cadf1c63e5ad5521a7bd88432d707d41"} Jul 14 00:53:04.520754 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.520722 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" event={"ID":"18bbc1d0-0722-40c4-b6aa-c400cc1882ed","Type":"ContainerStarted","Data":"54b825157f2cab5bc842b7aed5749fac9d9293daa9eb5aef24b93c688a88cbc8"} Jul 14 00:53:04.680694 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.680631 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:04.834719 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:04.834640 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5"] Jul 14 00:53:04.837202 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:53:04.837166 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8732d060_6b69_4aed_b89a_e836f3ad3a3a.slice/crio-7f243b3ee0d2e5968b0d308258b4b47ef0a2f82470017940124174a585c1bdbb WatchSource:0}: Error finding container 7f243b3ee0d2e5968b0d308258b4b47ef0a2f82470017940124174a585c1bdbb: Status 404 returned error can't find the container with id 7f243b3ee0d2e5968b0d308258b4b47ef0a2f82470017940124174a585c1bdbb Jul 14 00:53:05.278274 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:05.278238 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:05.531262 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:05.531179 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" event={"ID":"8732d060-6b69-4aed-b89a-e836f3ad3a3a","Type":"ContainerStarted","Data":"7f243b3ee0d2e5968b0d308258b4b47ef0a2f82470017940124174a585c1bdbb"} Jul 14 00:53:07.326259 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:07.326184 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:08.553563 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:08.553298 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" event={"ID":"ea299afa-3425-445e-817f-469542f41680","Type":"ContainerStarted","Data":"6ad208f576b2dac9db6c58af0419e7d8943012fb9565486d8cc3a31959a64899"} Jul 14 00:53:08.553563 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:08.553473 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" Jul 14 00:53:08.555051 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:08.555022 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" event={"ID":"18bbc1d0-0722-40c4-b6aa-c400cc1882ed","Type":"ContainerStarted","Data":"34d578cfade183119753efb1b6f3207ecfca5c7787fc6e841cb769d3fda5955e"} Jul 14 00:53:08.555205 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:08.555076 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" Jul 14 00:53:08.556653 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:08.556626 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" event={"ID":"a656bf6b-2f6c-4e6d-aee0-998cb9aafc22","Type":"ContainerStarted","Data":"ee17c0c8d47af545d85bb759659bd8735b431acc363548f1b9bdf910b53c8645"} Jul 14 00:53:08.569485 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:08.569435 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" podStartSLOduration=1.786733984 podStartE2EDuration="5.569417405s" totalImagesPullingTime="3.782683421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:53:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:53:08.567036291 +0000 UTC m=+525.418403662" watchObservedRunningTime="2026-07-14 00:53:08.569417405 +0000 UTC m=+525.420784777" Jul 14 00:53:08.582799 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:08.582757 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" podStartSLOduration=2.499014779 podStartE2EDuration="6.582744051s" totalImagesPullingTime="4.083729272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:53:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:53:08.580685048 +0000 UTC m=+525.432052410" watchObservedRunningTime="2026-07-14 00:53:08.582744051 +0000 UTC m=+525.434111420" Jul 14 00:53:08.598996 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:08.598923 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5f876ccc94-tgk5r" podStartSLOduration=1.653572555 podStartE2EDuration="6.598904337s" totalImagesPullingTime="4.945331782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:53:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:53:08.596099517 +0000 UTC m=+525.447466887" watchObservedRunningTime="2026-07-14 00:53:08.598904337 +0000 UTC m=+525.450271710" Jul 14 00:53:09.254085 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:09.254042 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:53:09.254217 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:09.254115 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:53:09.562814 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:09.562776 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" event={"ID":"8732d060-6b69-4aed-b89a-e836f3ad3a3a","Type":"ContainerStarted","Data":"0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e"} Jul 14 00:53:09.615358 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:09.615311 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" podStartSLOduration=1.201443642 podStartE2EDuration="5.61529647s" totalImagesPullingTime="4.413852828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:53:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:53:09.612493464 +0000 UTC m=+526.463860849" watchObservedRunningTime="2026-07-14 00:53:09.61529647 +0000 UTC m=+526.466663840" Jul 14 00:53:10.414794 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.414759 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:10.485315 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.485284 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:10.569345 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.569318 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:10.571119 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.571088 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-5fhp5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 00:53:10.571250 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.571164 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" podUID="8732d060-6b69-4aed-b89a-e836f3ad3a3a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 00:53:10.740182 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.740063 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp"] Jul 14 00:53:10.743412 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.743390 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.758017 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.757996 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-6tk4p\"" Jul 14 00:53:10.793645 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.793618 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp"] Jul 14 00:53:10.857741 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.857712 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.857887 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.857750 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.857887 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.857787 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.857887 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.857857 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.858041 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.857911 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.858041 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.857944 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.858041 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.857967 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.858185 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.858036 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.858185 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.858103 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wprch\" (UniqueName: \"kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-kube-api-access-wprch\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.875173 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.875137 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:10.947671 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.947640 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5"] Jul 14 00:53:10.950893 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.950875 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:10.959780 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.959747 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.959899 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.959838 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.959899 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.959876 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.960015 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.959907 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.960015 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.959947 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.960015 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.959987 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wprch\" (UniqueName: \"kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-kube-api-access-wprch\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.960166 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.960045 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.960166 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.960078 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.960166 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.960131 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.961570 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.960586 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-data\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.961570 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.961233 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.961570 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.961258 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.961570 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.961446 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.961570 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.961535 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.965611 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.964243 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.965712 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.965651 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.968550 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.968529 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5"] Jul 14 00:53:10.982560 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.982535 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wprch\" (UniqueName: \"kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-kube-api-access-wprch\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:10.982750 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:10.982728 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-token\") pod \"data-science-gateway-data-science-gateway-class-548557588-4ttgp\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:11.052698 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.052673 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:11.060945 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.060922 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.061041 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.060959 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d1d8abe-5a15-48cd-a639-5141082cc001-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.061041 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.060982 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.061041 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.061002 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.061236 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.061070 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr9x4\" (UniqueName: \"kubernetes.io/projected/9d1d8abe-5a15-48cd-a639-5141082cc001-kube-api-access-hr9x4\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.061236 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.061111 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.061236 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.061169 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.061236 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.061218 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.061436 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.061248 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.162280 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.161819 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.162280 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.161896 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.162280 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.161949 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d1d8abe-5a15-48cd-a639-5141082cc001-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.162280 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.161994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.162280 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.162030 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.162280 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.162071 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr9x4\" (UniqueName: \"kubernetes.io/projected/9d1d8abe-5a15-48cd-a639-5141082cc001-kube-api-access-hr9x4\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.162280 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.162105 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.163186 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.162831 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.163186 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.162883 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.163186 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.162969 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d1d8abe-5a15-48cd-a639-5141082cc001-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.163797 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.163446 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.164059 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.164008 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.164365 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.164283 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.164742 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.164711 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.165595 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.165574 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.166239 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.166220 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.170421 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.170398 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d1d8abe-5a15-48cd-a639-5141082cc001-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.170575 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.170559 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr9x4\" (UniqueName: \"kubernetes.io/projected/9d1d8abe-5a15-48cd-a639-5141082cc001-kube-api-access-hr9x4\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wz6h5\" (UID: \"9d1d8abe-5a15-48cd-a639-5141082cc001\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.189351 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.189324 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp"] Jul 14 00:53:11.193019 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:53:11.192991 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e92826_e5d7_4d89_b726_dde68ca2ff82.slice/crio-f35342c60544b977372f06698abdd8b302a8e10910060bf6d9d49fc289b7fe5d WatchSource:0}: Error finding container f35342c60544b977372f06698abdd8b302a8e10910060bf6d9d49fc289b7fe5d: Status 404 returned error can't find the container with id f35342c60544b977372f06698abdd8b302a8e10910060bf6d9d49fc289b7fe5d Jul 14 00:53:11.229329 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.229305 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:11.261810 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.261786 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:11.423464 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.423323 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5"] Jul 14 00:53:11.425727 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:53:11.425702 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1d8abe_5a15_48cd_a639_5141082cc001.slice/crio-12ff502eb7cbbdea0e94b7172a4b89ff32c20e1fc70eee3c9ae0d92ceabbe232 WatchSource:0}: Error finding container 12ff502eb7cbbdea0e94b7172a4b89ff32c20e1fc70eee3c9ae0d92ceabbe232: Status 404 returned error can't find the container with id 12ff502eb7cbbdea0e94b7172a4b89ff32c20e1fc70eee3c9ae0d92ceabbe232 Jul 14 00:53:11.427081 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.427058 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:11.574247 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.574169 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" event={"ID":"c5e92826-e5d7-4d89-b726-dde68ca2ff82","Type":"ContainerStarted","Data":"f35342c60544b977372f06698abdd8b302a8e10910060bf6d9d49fc289b7fe5d"} Jul 14 00:53:11.575443 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.575422 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" event={"ID":"9d1d8abe-5a15-48cd-a639-5141082cc001","Type":"ContainerStarted","Data":"12ff502eb7cbbdea0e94b7172a4b89ff32c20e1fc70eee3c9ae0d92ceabbe232"} Jul 14 00:53:11.576215 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:11.576183 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:12.937861 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:12.937816 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:12.997245 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:12.997217 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr"] Jul 14 00:53:13.001450 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.001425 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:53:13.003836 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.003356 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 14 00:53:13.003836 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.003675 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2q2h9\"" Jul 14 00:53:13.010072 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.009165 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz"] Jul 14 00:53:13.018609 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.017706 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr"] Jul 14 00:53:13.018609 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.017820 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" Jul 14 00:53:13.022413 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.020979 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-n6fqx\"" Jul 14 00:53:13.022413 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.021068 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 14 00:53:13.027117 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.027096 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz"] Jul 14 00:53:13.039166 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.039132 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:13.080614 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.080528 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t67w9\" (UniqueName: \"kubernetes.io/projected/db690295-f070-4975-ad18-bbb2e599affb-kube-api-access-t67w9\") pod \"kserve-controller-manager-d6f4b65b8-gkcmr\" (UID: \"db690295-f070-4975-ad18-bbb2e599affb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:53:13.080614 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.080584 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a592437-0608-4a2a-9e28-09ab2395b241-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bsdwz\" (UID: \"8a592437-0608-4a2a-9e28-09ab2395b241\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" Jul 14 00:53:13.080794 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.080623 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44bh4\" (UniqueName: \"kubernetes.io/projected/8a592437-0608-4a2a-9e28-09ab2395b241-kube-api-access-44bh4\") pod \"llmisvc-controller-manager-777c6bd7d5-bsdwz\" (UID: \"8a592437-0608-4a2a-9e28-09ab2395b241\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" Jul 14 00:53:13.080794 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.080680 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db690295-f070-4975-ad18-bbb2e599affb-cert\") pod \"kserve-controller-manager-d6f4b65b8-gkcmr\" (UID: \"db690295-f070-4975-ad18-bbb2e599affb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:53:13.181971 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.181906 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db690295-f070-4975-ad18-bbb2e599affb-cert\") pod \"kserve-controller-manager-d6f4b65b8-gkcmr\" (UID: \"db690295-f070-4975-ad18-bbb2e599affb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:53:13.182178 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.182042 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t67w9\" (UniqueName: \"kubernetes.io/projected/db690295-f070-4975-ad18-bbb2e599affb-kube-api-access-t67w9\") pod \"kserve-controller-manager-d6f4b65b8-gkcmr\" (UID: \"db690295-f070-4975-ad18-bbb2e599affb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:53:13.182178 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.182088 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a592437-0608-4a2a-9e28-09ab2395b241-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bsdwz\" (UID: \"8a592437-0608-4a2a-9e28-09ab2395b241\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" Jul 14 00:53:13.182178 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.182127 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44bh4\" (UniqueName: \"kubernetes.io/projected/8a592437-0608-4a2a-9e28-09ab2395b241-kube-api-access-44bh4\") pod \"llmisvc-controller-manager-777c6bd7d5-bsdwz\" (UID: \"8a592437-0608-4a2a-9e28-09ab2395b241\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" Jul 14 00:53:13.182345 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:53:13.182045 2557 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 14 00:53:13.182552 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:53:13.182398 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db690295-f070-4975-ad18-bbb2e599affb-cert podName:db690295-f070-4975-ad18-bbb2e599affb nodeName:}" failed. No retries permitted until 2026-07-14 00:53:13.682375899 +0000 UTC m=+530.533743252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/db690295-f070-4975-ad18-bbb2e599affb-cert") pod "kserve-controller-manager-d6f4b65b8-gkcmr" (UID: "db690295-f070-4975-ad18-bbb2e599affb") : secret "kserve-webhook-server-cert" not found Jul 14 00:53:13.182552 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:53:13.182551 2557 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 14 00:53:13.182759 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:53:13.182600 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a592437-0608-4a2a-9e28-09ab2395b241-cert podName:8a592437-0608-4a2a-9e28-09ab2395b241 nodeName:}" failed. No retries permitted until 2026-07-14 00:53:13.682582676 +0000 UTC m=+530.533950033 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8a592437-0608-4a2a-9e28-09ab2395b241-cert") pod "llmisvc-controller-manager-777c6bd7d5-bsdwz" (UID: "8a592437-0608-4a2a-9e28-09ab2395b241") : secret "llmisvc-webhook-server-cert" not found Jul 14 00:53:13.191241 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.191143 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t67w9\" (UniqueName: \"kubernetes.io/projected/db690295-f070-4975-ad18-bbb2e599affb-kube-api-access-t67w9\") pod \"kserve-controller-manager-d6f4b65b8-gkcmr\" (UID: \"db690295-f070-4975-ad18-bbb2e599affb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:53:13.192832 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.192807 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44bh4\" (UniqueName: \"kubernetes.io/projected/8a592437-0608-4a2a-9e28-09ab2395b241-kube-api-access-44bh4\") pod \"llmisvc-controller-manager-777c6bd7d5-bsdwz\" (UID: \"8a592437-0608-4a2a-9e28-09ab2395b241\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" Jul 14 00:53:13.205537 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.205509 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:13.329806 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.329760 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:13.460013 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.459922 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:13.687267 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.687238 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a592437-0608-4a2a-9e28-09ab2395b241-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bsdwz\" (UID: \"8a592437-0608-4a2a-9e28-09ab2395b241\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" Jul 14 00:53:13.687432 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.687307 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db690295-f070-4975-ad18-bbb2e599affb-cert\") pod \"kserve-controller-manager-d6f4b65b8-gkcmr\" (UID: \"db690295-f070-4975-ad18-bbb2e599affb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:53:13.687484 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:53:13.687452 2557 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 14 00:53:13.687518 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:53:13.687507 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/db690295-f070-4975-ad18-bbb2e599affb-cert podName:db690295-f070-4975-ad18-bbb2e599affb nodeName:}" failed. No retries permitted until 2026-07-14 00:53:14.687491387 +0000 UTC m=+531.538858740 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/db690295-f070-4975-ad18-bbb2e599affb-cert") pod "kserve-controller-manager-d6f4b65b8-gkcmr" (UID: "db690295-f070-4975-ad18-bbb2e599affb") : secret "kserve-webhook-server-cert" not found Jul 14 00:53:13.690716 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.690688 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a592437-0608-4a2a-9e28-09ab2395b241-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bsdwz\" (UID: \"8a592437-0608-4a2a-9e28-09ab2395b241\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" Jul 14 00:53:13.794534 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.794497 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:13.932875 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:13.932845 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" Jul 14 00:53:14.128959 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.128880 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5"] Jul 14 00:53:14.129399 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.129045 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" podUID="8732d060-6b69-4aed-b89a-e836f3ad3a3a" containerName="discovery" containerID="cri-o://0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e" gracePeriod=30 Jul 14 00:53:14.576365 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.576336 2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-5fhp5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.27:8080/ready\": dial tcp 10.134.0.27:8080: connect: connection refused" start-of-body= Jul 14 00:53:14.576521 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.576389 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" podUID="8732d060-6b69-4aed-b89a-e836f3ad3a3a" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.27:8080/ready\": dial tcp 10.134.0.27:8080: connect: connection refused" Jul 14 00:53:14.695927 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.695894 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db690295-f070-4975-ad18-bbb2e599affb-cert\") pod \"kserve-controller-manager-d6f4b65b8-gkcmr\" (UID: \"db690295-f070-4975-ad18-bbb2e599affb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:53:14.699035 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.698999 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db690295-f070-4975-ad18-bbb2e599affb-cert\") pod \"kserve-controller-manager-d6f4b65b8-gkcmr\" (UID: \"db690295-f070-4975-ad18-bbb2e599affb\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:53:14.818977 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.818947 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:53:14.939065 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.939014 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:53:14.939286 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.939109 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:53:14.939286 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.939188 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:53:14.948410 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.948377 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:53:14.948525 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.948457 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:53:14.948525 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:14.948503 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:53:15.018679 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.018654 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:15.041799 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.041558 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr"] Jul 14 00:53:15.045140 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:53:15.045102 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb690295_f070_4975_ad18_bbb2e599affb.slice/crio-121aa2bb921827eab5efa0fc93069b14c1df929ee1cc25461cd559ac6b524069 WatchSource:0}: Error finding container 121aa2bb921827eab5efa0fc93069b14c1df929ee1cc25461cd559ac6b524069: Status 404 returned error can't find the container with id 121aa2bb921827eab5efa0fc93069b14c1df929ee1cc25461cd559ac6b524069 Jul 14 00:53:15.065727 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.065666 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz"] Jul 14 00:53:15.074641 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:53:15.073464 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8a592437_0608_4a2a_9e28_09ab2395b241.slice/crio-0cb5ef729576decf8f9c7af35088efab45cdd57eab5c89884ec495e2d30d9ad1 WatchSource:0}: Error finding container 0cb5ef729576decf8f9c7af35088efab45cdd57eab5c89884ec495e2d30d9ad1: Status 404 returned error can't find the container with id 0cb5ef729576decf8f9c7af35088efab45cdd57eab5c89884ec495e2d30d9ad1 Jul 14 00:53:15.101226 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.101199 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-kubeconfig\") pod \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " Jul 14 00:53:15.101338 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.101265 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8732d060-6b69-4aed-b89a-e836f3ad3a3a-local-certs\") pod \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " Jul 14 00:53:15.101338 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.101298 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-dns-cert\") pod \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " Jul 14 00:53:15.101446 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.101358 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmt7k\" (UniqueName: \"kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-kube-api-access-fmt7k\") pod \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " Jul 14 00:53:15.101446 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.101406 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-cacerts\") pod \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " Jul 14 00:53:15.101446 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.101433 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-ca-configmap\") pod \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " Jul 14 00:53:15.101594 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.101492 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-token\") pod \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\" (UID: \"8732d060-6b69-4aed-b89a-e836f3ad3a3a\") " Jul 14 00:53:15.102838 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.102797 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "8732d060-6b69-4aed-b89a-e836f3ad3a3a" (UID: "8732d060-6b69-4aed-b89a-e836f3ad3a3a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 00:53:15.104943 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.104892 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "8732d060-6b69-4aed-b89a-e836f3ad3a3a" (UID: "8732d060-6b69-4aed-b89a-e836f3ad3a3a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 00:53:15.105668 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.105631 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-token" (OuterVolumeSpecName: "istio-token") pod "8732d060-6b69-4aed-b89a-e836f3ad3a3a" (UID: "8732d060-6b69-4aed-b89a-e836f3ad3a3a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:53:15.106022 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.105925 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8732d060-6b69-4aed-b89a-e836f3ad3a3a-local-certs" (OuterVolumeSpecName: "local-certs") pod "8732d060-6b69-4aed-b89a-e836f3ad3a3a" (UID: "8732d060-6b69-4aed-b89a-e836f3ad3a3a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:53:15.106022 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.105996 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "8732d060-6b69-4aed-b89a-e836f3ad3a3a" (UID: "8732d060-6b69-4aed-b89a-e836f3ad3a3a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 00:53:15.106566 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.106535 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-cacerts" (OuterVolumeSpecName: "cacerts") pod "8732d060-6b69-4aed-b89a-e836f3ad3a3a" (UID: "8732d060-6b69-4aed-b89a-e836f3ad3a3a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 00:53:15.107485 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.107460 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-kube-api-access-fmt7k" (OuterVolumeSpecName: "kube-api-access-fmt7k") pod "8732d060-6b69-4aed-b89a-e836f3ad3a3a" (UID: "8732d060-6b69-4aed-b89a-e836f3ad3a3a"). InnerVolumeSpecName "kube-api-access-fmt7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:53:15.202257 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.202192 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:15.202257 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.202217 2557 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-kubeconfig\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:15.202257 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.202227 2557 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8732d060-6b69-4aed-b89a-e836f3ad3a3a-local-certs\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:15.202257 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.202235 2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-dns-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:15.202257 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.202244 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmt7k\" (UniqueName: \"kubernetes.io/projected/8732d060-6b69-4aed-b89a-e836f3ad3a3a-kube-api-access-fmt7k\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:15.202257 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.202255 2557 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8732d060-6b69-4aed-b89a-e836f3ad3a3a-cacerts\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:15.202715 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.202267 2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8732d060-6b69-4aed-b89a-e836f3ad3a3a-istio-csr-ca-configmap\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:15.598091 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.598050 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" event={"ID":"c5e92826-e5d7-4d89-b726-dde68ca2ff82","Type":"ContainerStarted","Data":"85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa"} Jul 14 00:53:15.599324 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.599301 2557 generic.go:356] "Generic (PLEG): container finished" podID="8732d060-6b69-4aed-b89a-e836f3ad3a3a" containerID="0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e" exitCode=0 Jul 14 00:53:15.599454 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.599337 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" Jul 14 00:53:15.599454 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.599373 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" event={"ID":"8732d060-6b69-4aed-b89a-e836f3ad3a3a","Type":"ContainerDied","Data":"0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e"} Jul 14 00:53:15.599454 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.599413 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5" event={"ID":"8732d060-6b69-4aed-b89a-e836f3ad3a3a","Type":"ContainerDied","Data":"7f243b3ee0d2e5968b0d308258b4b47ef0a2f82470017940124174a585c1bdbb"} Jul 14 00:53:15.599454 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.599435 2557 scope.go:117] "RemoveContainer" containerID="0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e" Jul 14 00:53:15.600713 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.600689 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" event={"ID":"db690295-f070-4975-ad18-bbb2e599affb","Type":"ContainerStarted","Data":"121aa2bb921827eab5efa0fc93069b14c1df929ee1cc25461cd559ac6b524069"} Jul 14 00:53:15.601818 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.601795 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" event={"ID":"8a592437-0608-4a2a-9e28-09ab2395b241","Type":"ContainerStarted","Data":"0cb5ef729576decf8f9c7af35088efab45cdd57eab5c89884ec495e2d30d9ad1"} Jul 14 00:53:15.603431 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.603405 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" event={"ID":"9d1d8abe-5a15-48cd-a639-5141082cc001","Type":"ContainerStarted","Data":"e2a19c3783d550061cf1b3694a189873c0a376bb37fc5538d4bc7ce023760c4f"} Jul 14 00:53:15.608084 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.608068 2557 scope.go:117] "RemoveContainer" containerID="0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e" Jul 14 00:53:15.608398 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:53:15.608378 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e\": container with ID starting with 0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e not found: ID does not exist" containerID="0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e" Jul 14 00:53:15.608464 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.608406 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e"} err="failed to get container status \"0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e\": rpc error: code = NotFound desc = could not find container \"0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e\": container with ID starting with 0222952b51f8d6942b9c4f65d5675bb61545a7311209f97ca3151aeda89f4c9e not found: ID does not exist" Jul 14 00:53:15.664776 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.664731 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" podStartSLOduration=1.911544418 podStartE2EDuration="5.664718589s" totalImagesPullingTime="3.753174171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:53:15.662911292 +0000 UTC m=+532.514278662" watchObservedRunningTime="2026-07-14 00:53:15.664718589 +0000 UTC m=+532.516085959" Jul 14 00:53:15.695546 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.695524 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5"] Jul 14 00:53:15.699502 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.699481 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-5fhp5"] Jul 14 00:53:15.739656 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:15.739618 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" podStartSLOduration=2.228650415 podStartE2EDuration="5.739603674s" totalImagesPullingTime="3.510953259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:53:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:53:15.737947696 +0000 UTC m=+532.589315065" watchObservedRunningTime="2026-07-14 00:53:15.739603674 +0000 UTC m=+532.590971045" Jul 14 00:53:16.053718 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:16.053685 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:16.055255 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:16.055227 2557 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-4ttgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 14 00:53:16.055406 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:16.055281 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" podUID="c5e92826-e5d7-4d89-b726-dde68ca2ff82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 14 00:53:16.262319 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:16.262270 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:16.268483 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:16.268456 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:16.610052 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:16.610023 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:16.611056 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:16.611032 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wz6h5" Jul 14 00:53:16.668666 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:16.668637 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp"] Jul 14 00:53:17.053160 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:17.053114 2557 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-4ttgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 14 00:53:17.053368 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:17.053196 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" podUID="c5e92826-e5d7-4d89-b726-dde68ca2ff82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 14 00:53:17.680397 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:17.680345 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8732d060-6b69-4aed-b89a-e836f3ad3a3a" path="/var/lib/kubelet/pods/8732d060-6b69-4aed-b89a-e836f3ad3a3a/volumes" Jul 14 00:53:18.053826 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:18.053791 2557 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-548557588-4ttgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 14 00:53:18.054002 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:18.053853 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" podUID="c5e92826-e5d7-4d89-b726-dde68ca2ff82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 14 00:53:18.621142 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:18.621103 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" event={"ID":"db690295-f070-4975-ad18-bbb2e599affb","Type":"ContainerStarted","Data":"8598194ee24ce1bf0aba691c32a47fdc7c1d9deaa9877cc1b68e97848d6dea20"} Jul 14 00:53:18.621319 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:18.621223 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" podUID="c5e92826-e5d7-4d89-b726-dde68ca2ff82" containerName="istio-proxy" containerID="cri-o://85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa" gracePeriod=30 Jul 14 00:53:18.689109 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:18.689068 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" podStartSLOduration=3.919598517 podStartE2EDuration="6.689053352s" totalImagesPullingTime="2.769454835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:53:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:53:18.688082818 +0000 UTC m=+535.539450189" watchObservedRunningTime="2026-07-14 00:53:18.689053352 +0000 UTC m=+535.540420722" Jul 14 00:53:19.565566 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:19.565540 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-kjxkv" Jul 14 00:53:19.567581 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:19.567562 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-c8skl" Jul 14 00:53:19.635046 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:19.635010 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" event={"ID":"8a592437-0608-4a2a-9e28-09ab2395b241","Type":"ContainerStarted","Data":"28cc0e7c1fe5905726a5f7932f3356c28da2766dcc916e68171b0d5426008c95"} Jul 14 00:53:19.635247 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:19.635099 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" Jul 14 00:53:19.635309 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:19.635248 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:53:19.651921 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:19.651879 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" podStartSLOduration=4.090937857 podStartE2EDuration="7.65185998s" totalImagesPullingTime="3.560922123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:53:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:53:19.650009409 +0000 UTC m=+536.501376779" watchObservedRunningTime="2026-07-14 00:53:19.65185998 +0000 UTC m=+536.503227351" Jul 14 00:53:21.760417 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:21.760384 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:21.860619 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:21.860591 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:23.507562 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.507520 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:23.559635 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.559607 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:53:23.861090 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.861071 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:23.973016 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.972981 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-certs\") pod \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " Jul 14 00:53:23.973135 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973037 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-credential-socket\") pod \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " Jul 14 00:53:23.973135 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973077 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-envoy\") pod \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " Jul 14 00:53:23.973135 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973109 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-socket\") pod \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " Jul 14 00:53:23.973259 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973201 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-data\") pod \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " Jul 14 00:53:23.973259 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973227 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istiod-ca-cert\") pod \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " Jul 14 00:53:23.973259 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973250 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-token\") pod \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " Jul 14 00:53:23.973387 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973285 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c5e92826-e5d7-4d89-b726-dde68ca2ff82" (UID: "c5e92826-e5d7-4d89-b726-dde68ca2ff82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:53:23.973387 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973294 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-podinfo\") pod \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " Jul 14 00:53:23.973387 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973317 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wprch\" (UniqueName: \"kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-kube-api-access-wprch\") pod \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\" (UID: \"c5e92826-e5d7-4d89-b726-dde68ca2ff82\") " Jul 14 00:53:23.973542 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973397 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c5e92826-e5d7-4d89-b726-dde68ca2ff82" (UID: "c5e92826-e5d7-4d89-b726-dde68ca2ff82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:53:23.973542 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973444 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-data" (OuterVolumeSpecName: "istio-data") pod "c5e92826-e5d7-4d89-b726-dde68ca2ff82" (UID: "c5e92826-e5d7-4d89-b726-dde68ca2ff82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:53:23.973542 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973458 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c5e92826-e5d7-4d89-b726-dde68ca2ff82" (UID: "c5e92826-e5d7-4d89-b726-dde68ca2ff82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:53:23.973694 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973660 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-data\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:23.973694 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973669 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c5e92826-e5d7-4d89-b726-dde68ca2ff82" (UID: "c5e92826-e5d7-4d89-b726-dde68ca2ff82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 00:53:23.973694 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973681 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-certs\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:23.973815 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973715 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-credential-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:23.973815 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.973734 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-workload-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:23.975357 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.975326 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-kube-api-access-wprch" (OuterVolumeSpecName: "kube-api-access-wprch") pod "c5e92826-e5d7-4d89-b726-dde68ca2ff82" (UID: "c5e92826-e5d7-4d89-b726-dde68ca2ff82"). InnerVolumeSpecName "kube-api-access-wprch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:53:23.975470 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.975372 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-token" (OuterVolumeSpecName: "istio-token") pod "c5e92826-e5d7-4d89-b726-dde68ca2ff82" (UID: "c5e92826-e5d7-4d89-b726-dde68ca2ff82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:53:23.975621 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.975599 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c5e92826-e5d7-4d89-b726-dde68ca2ff82" (UID: "c5e92826-e5d7-4d89-b726-dde68ca2ff82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:53:23.975777 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:23.975758 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c5e92826-e5d7-4d89-b726-dde68ca2ff82" (UID: "c5e92826-e5d7-4d89-b726-dde68ca2ff82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 00:53:24.074293 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.074236 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-podinfo\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:24.074293 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.074265 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wprch\" (UniqueName: \"kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-kube-api-access-wprch\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:24.074293 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.074275 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-envoy\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:24.074293 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.074284 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istiod-ca-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:24.074293 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.074292 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5e92826-e5d7-4d89-b726-dde68ca2ff82-istio-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:53:24.659957 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.659920 2557 generic.go:356] "Generic (PLEG): container finished" podID="c5e92826-e5d7-4d89-b726-dde68ca2ff82" containerID="85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa" exitCode=0 Jul 14 00:53:24.660438 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.659965 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" event={"ID":"c5e92826-e5d7-4d89-b726-dde68ca2ff82","Type":"ContainerDied","Data":"85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa"} Jul 14 00:53:24.660438 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.659980 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" Jul 14 00:53:24.660438 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.660029 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp" event={"ID":"c5e92826-e5d7-4d89-b726-dde68ca2ff82","Type":"ContainerDied","Data":"f35342c60544b977372f06698abdd8b302a8e10910060bf6d9d49fc289b7fe5d"} Jul 14 00:53:24.660438 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.660066 2557 scope.go:117] "RemoveContainer" containerID="85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa" Jul 14 00:53:24.669784 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.669763 2557 scope.go:117] "RemoveContainer" containerID="85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa" Jul 14 00:53:24.670097 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:53:24.670073 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa\": container with ID starting with 85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa not found: ID does not exist" containerID="85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa" Jul 14 00:53:24.670174 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.670107 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa"} err="failed to get container status \"85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa\": rpc error: code = NotFound desc = could not find container \"85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa\": container with ID starting with 85545a50a62fd4921c2ce8803f4e3183a7374eedf302525c3759098b98dd84fa not found: ID does not exist" Jul 14 00:53:24.681431 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.681402 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp"] Jul 14 00:53:24.685426 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:24.685405 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-548557588-4ttgp"] Jul 14 00:53:25.677662 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:25.677629 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5e92826-e5d7-4d89-b726-dde68ca2ff82" path="/var/lib/kubelet/pods/c5e92826-e5d7-4d89-b726-dde68ca2ff82/volumes" Jul 14 00:53:50.643309 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:50.643268 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bsdwz" Jul 14 00:53:50.646452 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:53:50.646430 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-gkcmr" Jul 14 00:54:03.271509 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.271472 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:03.490445 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.490409 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:03.600770 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.600747 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:03.697680 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.697653 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqb8r"] Jul 14 00:54:03.698016 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.698000 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8732d060-6b69-4aed-b89a-e836f3ad3a3a" containerName="discovery" Jul 14 00:54:03.698098 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.698018 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8732d060-6b69-4aed-b89a-e836f3ad3a3a" containerName="discovery" Jul 14 00:54:03.698098 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.698029 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e92826-e5d7-4d89-b726-dde68ca2ff82" containerName="istio-proxy" Jul 14 00:54:03.698098 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.698037 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e92826-e5d7-4d89-b726-dde68ca2ff82" containerName="istio-proxy" Jul 14 00:54:03.698276 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.698110 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e92826-e5d7-4d89-b726-dde68ca2ff82" containerName="istio-proxy" Jul 14 00:54:03.698276 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.698123 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8732d060-6b69-4aed-b89a-e836f3ad3a3a" containerName="discovery" Jul 14 00:54:03.702576 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.702554 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" Jul 14 00:54:03.705035 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.705014 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 14 00:54:03.705190 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.705040 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 14 00:54:03.705190 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.705108 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-cv4zf\"" Jul 14 00:54:03.712278 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.712253 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqb8r"] Jul 14 00:54:03.749172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.749136 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:03.854479 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.854426 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mm2d\" (UniqueName: \"kubernetes.io/projected/dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924-kube-api-access-7mm2d\") pod \"kuadrant-operator-catalog-dqb8r\" (UID: \"dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924\") " pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" Jul 14 00:54:03.955015 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.954992 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mm2d\" (UniqueName: \"kubernetes.io/projected/dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924-kube-api-access-7mm2d\") pod \"kuadrant-operator-catalog-dqb8r\" (UID: \"dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924\") " pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" Jul 14 00:54:03.965541 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:03.965511 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mm2d\" (UniqueName: \"kubernetes.io/projected/dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924-kube-api-access-7mm2d\") pod \"kuadrant-operator-catalog-dqb8r\" (UID: \"dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924\") " pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" Jul 14 00:54:04.012887 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.012862 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" Jul 14 00:54:04.013669 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.013650 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqb8r"] Jul 14 00:54:04.139302 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.139271 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqb8r"] Jul 14 00:54:04.141671 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:54:04.141643 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd1dbb80_3f8f_48fd_9cb3_c68c5f8e9924.slice/crio-275b0f5c5459d30766a3105f21ce39b29d23ed958067fa663b19c5e90222946b WatchSource:0}: Error finding container 275b0f5c5459d30766a3105f21ce39b29d23ed958067fa663b19c5e90222946b: Status 404 returned error can't find the container with id 275b0f5c5459d30766a3105f21ce39b29d23ed958067fa663b19c5e90222946b Jul 14 00:54:04.219542 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.219510 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n7rkg"] Jul 14 00:54:04.222257 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.222239 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" Jul 14 00:54:04.229522 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.229263 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n7rkg"] Jul 14 00:54:04.357792 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.357769 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swnng\" (UniqueName: \"kubernetes.io/projected/3cb21005-3f32-45ac-ae4c-5b862656a112-kube-api-access-swnng\") pod \"kuadrant-operator-catalog-n7rkg\" (UID: \"3cb21005-3f32-45ac-ae4c-5b862656a112\") " pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" Jul 14 00:54:04.458281 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.458227 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swnng\" (UniqueName: \"kubernetes.io/projected/3cb21005-3f32-45ac-ae4c-5b862656a112-kube-api-access-swnng\") pod \"kuadrant-operator-catalog-n7rkg\" (UID: \"3cb21005-3f32-45ac-ae4c-5b862656a112\") " pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" Jul 14 00:54:04.466496 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.466478 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swnng\" (UniqueName: \"kubernetes.io/projected/3cb21005-3f32-45ac-ae4c-5b862656a112-kube-api-access-swnng\") pod \"kuadrant-operator-catalog-n7rkg\" (UID: \"3cb21005-3f32-45ac-ae4c-5b862656a112\") " pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" Jul 14 00:54:04.479410 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.479378 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:04.532648 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.532630 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" Jul 14 00:54:04.650841 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.650814 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:04.656585 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.656565 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-n7rkg"] Jul 14 00:54:04.658680 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:54:04.658655 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cb21005_3f32_45ac_ae4c_5b862656a112.slice/crio-cb6eebcad0e4abe69ff317287fe21e8efee5cf7714bed9e6cfd86d41f8c0e3da WatchSource:0}: Error finding container cb6eebcad0e4abe69ff317287fe21e8efee5cf7714bed9e6cfd86d41f8c0e3da: Status 404 returned error can't find the container with id cb6eebcad0e4abe69ff317287fe21e8efee5cf7714bed9e6cfd86d41f8c0e3da Jul 14 00:54:04.840107 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.840077 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" event={"ID":"3cb21005-3f32-45ac-ae4c-5b862656a112","Type":"ContainerStarted","Data":"cb6eebcad0e4abe69ff317287fe21e8efee5cf7714bed9e6cfd86d41f8c0e3da"} Jul 14 00:54:04.841115 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:04.841092 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" event={"ID":"dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924","Type":"ContainerStarted","Data":"275b0f5c5459d30766a3105f21ce39b29d23ed958067fa663b19c5e90222946b"} Jul 14 00:54:07.859905 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:07.859861 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" event={"ID":"3cb21005-3f32-45ac-ae4c-5b862656a112","Type":"ContainerStarted","Data":"aa96b5bf3b5d2cd746a5b53eda710cd519f5eda17c2aa1c04c28addb772ca176"} Jul 14 00:54:07.861796 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:07.861765 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" event={"ID":"dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924","Type":"ContainerStarted","Data":"233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d"} Jul 14 00:54:07.861933 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:07.861812 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" podUID="dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924" containerName="registry-server" containerID="cri-o://233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d" gracePeriod=2 Jul 14 00:54:07.873403 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:07.873344 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" podStartSLOduration=1.692200462 podStartE2EDuration="3.873325192s" totalImagesPullingTime="2.18112473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:54:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:54:07.872263825 +0000 UTC m=+584.723631194" watchObservedRunningTime="2026-07-14 00:54:07.873325192 +0000 UTC m=+584.724692546" Jul 14 00:54:07.886593 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:07.886542 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" podStartSLOduration=2.19191944 podStartE2EDuration="4.886526188s" totalImagesPullingTime="2.694606748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:54:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:54:07.885998671 +0000 UTC m=+584.737366042" watchObservedRunningTime="2026-07-14 00:54:07.886526188 +0000 UTC m=+584.737893559" Jul 14 00:54:08.110101 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.110044 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" Jul 14 00:54:08.288984 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.288953 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7mm2d\" (UniqueName: \"kubernetes.io/projected/dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924-kube-api-access-7mm2d\") pod \"dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924\" (UID: \"dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924\") " Jul 14 00:54:08.291078 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.291053 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924-kube-api-access-7mm2d" (OuterVolumeSpecName: "kube-api-access-7mm2d") pod "dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924" (UID: "dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924"). InnerVolumeSpecName "kube-api-access-7mm2d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:54:08.390042 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.389982 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7mm2d\" (UniqueName: \"kubernetes.io/projected/dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924-kube-api-access-7mm2d\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:08.867847 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.867815 2557 generic.go:356] "Generic (PLEG): container finished" podID="dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924" containerID="233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d" exitCode=0 Jul 14 00:54:08.868338 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.867864 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" Jul 14 00:54:08.868338 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.867910 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" event={"ID":"dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924","Type":"ContainerDied","Data":"233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d"} Jul 14 00:54:08.868338 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.867979 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dqb8r" event={"ID":"dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924","Type":"ContainerDied","Data":"275b0f5c5459d30766a3105f21ce39b29d23ed958067fa663b19c5e90222946b"} Jul 14 00:54:08.868338 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.868014 2557 scope.go:117] "RemoveContainer" containerID="233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d" Jul 14 00:54:08.878569 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.878543 2557 scope.go:117] "RemoveContainer" containerID="233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d" Jul 14 00:54:08.878859 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:54:08.878826 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d\": container with ID starting with 233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d not found: ID does not exist" containerID="233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d" Jul 14 00:54:08.879004 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.878868 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d"} err="failed to get container status \"233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d\": rpc error: code = NotFound desc = could not find container \"233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d\": container with ID starting with 233c82c547b4c5779ec7c48eba241b3408568b116dd5de1ab286a5d31ffb1e1d not found: ID does not exist" Jul 14 00:54:08.888246 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.888215 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqb8r"] Jul 14 00:54:08.889849 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:08.889826 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dqb8r"] Jul 14 00:54:09.677996 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:09.677966 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924" path="/var/lib/kubelet/pods/dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924/volumes" Jul 14 00:54:14.533125 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:14.533087 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" Jul 14 00:54:14.533568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:14.533163 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" Jul 14 00:54:14.558117 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:14.558090 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" Jul 14 00:54:14.846871 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:14.846804 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:14.917902 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:14.917879 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-n7rkg" Jul 14 00:54:19.255171 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.255123 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php"] Jul 14 00:54:19.255681 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.255646 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924" containerName="registry-server" Jul 14 00:54:19.255681 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.255668 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924" containerName="registry-server" Jul 14 00:54:19.255798 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.255784 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd1dbb80-3f8f-48fd-9cb3-c68c5f8e9924" containerName="registry-server" Jul 14 00:54:19.259228 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.259200 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:19.261399 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.261369 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-n2brf\"" Jul 14 00:54:19.266790 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.266748 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php"] Jul 14 00:54:19.356863 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.356832 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:19.356975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.356890 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgjnl\" (UniqueName: \"kubernetes.io/projected/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-kube-api-access-jgjnl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:19.356975 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.356913 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:19.457389 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.457356 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:19.457544 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.457518 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgjnl\" (UniqueName: \"kubernetes.io/projected/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-kube-api-access-jgjnl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:19.457599 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.457566 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:19.457757 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.457737 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:19.457866 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.457849 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:19.465683 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.465657 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgjnl\" (UniqueName: \"kubernetes.io/projected/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-kube-api-access-jgjnl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:19.570021 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.569996 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:19.655603 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.655571 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz"] Jul 14 00:54:19.660585 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.660563 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:19.667382 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.667362 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz"] Jul 14 00:54:19.706800 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.706773 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php"] Jul 14 00:54:19.709075 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:54:19.709053 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb1c7be4_2ea5_4cb0_9dd0_e440bae11050.slice/crio-2cf6e0a2b47b689ed36fae2a1c0392882cec113555d7654373f19870efa194fc WatchSource:0}: Error finding container 2cf6e0a2b47b689ed36fae2a1c0392882cec113555d7654373f19870efa194fc: Status 404 returned error can't find the container with id 2cf6e0a2b47b689ed36fae2a1c0392882cec113555d7654373f19870efa194fc Jul 14 00:54:19.760342 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.760318 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:19.760438 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.760358 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:19.760438 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.760387 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m664p\" (UniqueName: \"kubernetes.io/projected/257eede3-e32d-4320-bf9d-b1b2f95b118d-kube-api-access-m664p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:19.860807 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.860753 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:19.860807 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.860784 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:19.860807 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.860805 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m664p\" (UniqueName: \"kubernetes.io/projected/257eede3-e32d-4320-bf9d-b1b2f95b118d-kube-api-access-m664p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:19.861096 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.861078 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:19.861143 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.861115 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:19.869247 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.869225 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m664p\" (UniqueName: \"kubernetes.io/projected/257eede3-e32d-4320-bf9d-b1b2f95b118d-kube-api-access-m664p\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:19.920926 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.920891 2557 generic.go:356] "Generic (PLEG): container finished" podID="db1c7be4-2ea5-4cb0-9dd0-e440bae11050" containerID="5f05f01139219e53e380f33e9ebfb2443bc4843e33ad342f58940182597cbc44" exitCode=0 Jul 14 00:54:19.921021 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.920938 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" event={"ID":"db1c7be4-2ea5-4cb0-9dd0-e440bae11050","Type":"ContainerDied","Data":"5f05f01139219e53e380f33e9ebfb2443bc4843e33ad342f58940182597cbc44"} Jul 14 00:54:19.921021 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.920977 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" event={"ID":"db1c7be4-2ea5-4cb0-9dd0-e440bae11050","Type":"ContainerStarted","Data":"2cf6e0a2b47b689ed36fae2a1c0392882cec113555d7654373f19870efa194fc"} Jul 14 00:54:19.972255 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:19.972229 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:20.057273 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.057243 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj"] Jul 14 00:54:20.061987 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.061966 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:20.068725 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.068576 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj"] Jul 14 00:54:20.101693 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.101673 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz"] Jul 14 00:54:20.103292 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:54:20.103265 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod257eede3_e32d_4320_bf9d_b1b2f95b118d.slice/crio-d6e58b120edeaf1057907abd6bf8ff5b8ee5cb33f70493e3fb15a99ec243e635 WatchSource:0}: Error finding container d6e58b120edeaf1057907abd6bf8ff5b8ee5cb33f70493e3fb15a99ec243e635: Status 404 returned error can't find the container with id d6e58b120edeaf1057907abd6bf8ff5b8ee5cb33f70493e3fb15a99ec243e635 Jul 14 00:54:20.163121 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.163102 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:20.163212 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.163146 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzx8n\" (UniqueName: \"kubernetes.io/projected/b1008438-9427-4355-83c6-e1084301e7dd-kube-api-access-zzx8n\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:20.163268 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.163224 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:20.264109 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.264082 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzx8n\" (UniqueName: \"kubernetes.io/projected/b1008438-9427-4355-83c6-e1084301e7dd-kube-api-access-zzx8n\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:20.264399 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.264128 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:20.264399 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.264325 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:20.264532 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.264514 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:20.264647 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.264632 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:20.272892 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.272868 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzx8n\" (UniqueName: \"kubernetes.io/projected/b1008438-9427-4355-83c6-e1084301e7dd-kube-api-access-zzx8n\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:20.372812 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.372767 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:20.510223 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.510201 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj"] Jul 14 00:54:20.512203 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:54:20.512178 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1008438_9427_4355_83c6_e1084301e7dd.slice/crio-83cd458025269229cadf2982ceaab23aa7f49d1692e2daa319392d2b9aab1e58 WatchSource:0}: Error finding container 83cd458025269229cadf2982ceaab23aa7f49d1692e2daa319392d2b9aab1e58: Status 404 returned error can't find the container with id 83cd458025269229cadf2982ceaab23aa7f49d1692e2daa319392d2b9aab1e58 Jul 14 00:54:20.536496 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.536474 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb"] Jul 14 00:54:20.540736 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.540721 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:20.548629 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.548608 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb"] Jul 14 00:54:20.565601 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.565581 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:20.565689 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.565634 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxpn9\" (UniqueName: \"kubernetes.io/projected/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-kube-api-access-sxpn9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:20.565727 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.565713 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:20.666324 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.666303 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxpn9\" (UniqueName: \"kubernetes.io/projected/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-kube-api-access-sxpn9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:20.666447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.666339 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:20.666447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.666385 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:20.666737 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.666718 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:20.666780 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.666742 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:20.674552 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.674530 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxpn9\" (UniqueName: \"kubernetes.io/projected/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-kube-api-access-sxpn9\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:20.865753 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.865726 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:20.929134 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.929105 2557 generic.go:356] "Generic (PLEG): container finished" podID="b1008438-9427-4355-83c6-e1084301e7dd" containerID="dae79c3a38e62ddecaddcc0d374a3afed6f9b288a153a96136dc992900d1e614" exitCode=0 Jul 14 00:54:20.929273 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.929197 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" event={"ID":"b1008438-9427-4355-83c6-e1084301e7dd","Type":"ContainerDied","Data":"dae79c3a38e62ddecaddcc0d374a3afed6f9b288a153a96136dc992900d1e614"} Jul 14 00:54:20.929273 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.929258 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" event={"ID":"b1008438-9427-4355-83c6-e1084301e7dd","Type":"ContainerStarted","Data":"83cd458025269229cadf2982ceaab23aa7f49d1692e2daa319392d2b9aab1e58"} Jul 14 00:54:20.930714 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.930694 2557 generic.go:356] "Generic (PLEG): container finished" podID="257eede3-e32d-4320-bf9d-b1b2f95b118d" containerID="b4d267f3e5c7e1aad921dc13cc1dac764036edfbc920b0e93c16ef2f7efe067b" exitCode=0 Jul 14 00:54:20.930812 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.930782 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" event={"ID":"257eede3-e32d-4320-bf9d-b1b2f95b118d","Type":"ContainerDied","Data":"b4d267f3e5c7e1aad921dc13cc1dac764036edfbc920b0e93c16ef2f7efe067b"} Jul 14 00:54:20.930855 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:20.930830 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" event={"ID":"257eede3-e32d-4320-bf9d-b1b2f95b118d","Type":"ContainerStarted","Data":"d6e58b120edeaf1057907abd6bf8ff5b8ee5cb33f70493e3fb15a99ec243e635"} Jul 14 00:54:21.001080 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:21.001050 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb"] Jul 14 00:54:21.004976 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:54:21.004949 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55f5baad_9e1e_478f_8ba1_4d1dc53a8527.slice/crio-dcfcdd1e2093c2e9f591122a35b48d9eb8f06417e233f6342aa5da3070847114 WatchSource:0}: Error finding container dcfcdd1e2093c2e9f591122a35b48d9eb8f06417e233f6342aa5da3070847114: Status 404 returned error can't find the container with id dcfcdd1e2093c2e9f591122a35b48d9eb8f06417e233f6342aa5da3070847114 Jul 14 00:54:21.937494 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:21.937469 2557 generic.go:356] "Generic (PLEG): container finished" podID="257eede3-e32d-4320-bf9d-b1b2f95b118d" containerID="3df27e9d98e13b49b647f8452396eb22a87eab6c4f54065c46bf3d6381e299ea" exitCode=0 Jul 14 00:54:21.937864 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:21.937550 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" event={"ID":"257eede3-e32d-4320-bf9d-b1b2f95b118d","Type":"ContainerDied","Data":"3df27e9d98e13b49b647f8452396eb22a87eab6c4f54065c46bf3d6381e299ea"} Jul 14 00:54:21.939666 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:21.939642 2557 generic.go:356] "Generic (PLEG): container finished" podID="db1c7be4-2ea5-4cb0-9dd0-e440bae11050" containerID="47a4e111a9ab37644795e4892f34ae2f91995c187ad39043dcefdc17a37f28a4" exitCode=0 Jul 14 00:54:21.939774 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:21.939735 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" event={"ID":"db1c7be4-2ea5-4cb0-9dd0-e440bae11050","Type":"ContainerDied","Data":"47a4e111a9ab37644795e4892f34ae2f91995c187ad39043dcefdc17a37f28a4"} Jul 14 00:54:21.941412 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:21.941393 2557 generic.go:356] "Generic (PLEG): container finished" podID="55f5baad-9e1e-478f-8ba1-4d1dc53a8527" containerID="e3998cb8d66087a7fecc919b3646cb519ae9613005213deed26d23cdca4129c0" exitCode=0 Jul 14 00:54:21.941506 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:21.941424 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" event={"ID":"55f5baad-9e1e-478f-8ba1-4d1dc53a8527","Type":"ContainerDied","Data":"e3998cb8d66087a7fecc919b3646cb519ae9613005213deed26d23cdca4129c0"} Jul 14 00:54:21.941506 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:21.941462 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" event={"ID":"55f5baad-9e1e-478f-8ba1-4d1dc53a8527","Type":"ContainerStarted","Data":"dcfcdd1e2093c2e9f591122a35b48d9eb8f06417e233f6342aa5da3070847114"} Jul 14 00:54:22.947802 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:22.947769 2557 generic.go:356] "Generic (PLEG): container finished" podID="db1c7be4-2ea5-4cb0-9dd0-e440bae11050" containerID="653017893a8b0c0eeb146bcb9b30dd2e40f98ea5ea4df6b30f1ac146ae429cad" exitCode=0 Jul 14 00:54:22.948193 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:22.947855 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" event={"ID":"db1c7be4-2ea5-4cb0-9dd0-e440bae11050","Type":"ContainerDied","Data":"653017893a8b0c0eeb146bcb9b30dd2e40f98ea5ea4df6b30f1ac146ae429cad"} Jul 14 00:54:22.949572 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:22.949550 2557 generic.go:356] "Generic (PLEG): container finished" podID="55f5baad-9e1e-478f-8ba1-4d1dc53a8527" containerID="56e8e3dfcae1bb6b12450da84bcbd1f26a32cd5b879b92447b07a9e8f24c3e0b" exitCode=0 Jul 14 00:54:22.949657 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:22.949630 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" event={"ID":"55f5baad-9e1e-478f-8ba1-4d1dc53a8527","Type":"ContainerDied","Data":"56e8e3dfcae1bb6b12450da84bcbd1f26a32cd5b879b92447b07a9e8f24c3e0b"} Jul 14 00:54:22.951601 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:22.951578 2557 generic.go:356] "Generic (PLEG): container finished" podID="b1008438-9427-4355-83c6-e1084301e7dd" containerID="04051a1db0b30708a7750d277d59abf7470eac8b2bfa266bde0b146cf5c18a8d" exitCode=0 Jul 14 00:54:22.951715 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:22.951650 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" event={"ID":"b1008438-9427-4355-83c6-e1084301e7dd","Type":"ContainerDied","Data":"04051a1db0b30708a7750d277d59abf7470eac8b2bfa266bde0b146cf5c18a8d"} Jul 14 00:54:22.953704 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:22.953685 2557 generic.go:356] "Generic (PLEG): container finished" podID="257eede3-e32d-4320-bf9d-b1b2f95b118d" containerID="25b869808d95ce894369adca799e9e80397b4b794be33e09038256d5206f3abc" exitCode=0 Jul 14 00:54:22.953778 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:22.953730 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" event={"ID":"257eede3-e32d-4320-bf9d-b1b2f95b118d","Type":"ContainerDied","Data":"25b869808d95ce894369adca799e9e80397b4b794be33e09038256d5206f3abc"} Jul 14 00:54:23.544197 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:23.544173 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 00:54:23.545239 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:23.545218 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 00:54:23.959915 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:23.959883 2557 generic.go:356] "Generic (PLEG): container finished" podID="55f5baad-9e1e-478f-8ba1-4d1dc53a8527" containerID="327d833f5ab7914e1fb63dd940cf02b391abaaef6cc6d705660216d5e9508ebc" exitCode=0 Jul 14 00:54:23.960294 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:23.959972 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" event={"ID":"55f5baad-9e1e-478f-8ba1-4d1dc53a8527","Type":"ContainerDied","Data":"327d833f5ab7914e1fb63dd940cf02b391abaaef6cc6d705660216d5e9508ebc"} Jul 14 00:54:23.961819 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:23.961796 2557 generic.go:356] "Generic (PLEG): container finished" podID="b1008438-9427-4355-83c6-e1084301e7dd" containerID="baf9d096dbc38752fbc488dc8d42bfb420c00073043fb7a3c6040479724d8b4b" exitCode=0 Jul 14 00:54:23.961925 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:23.961827 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" event={"ID":"b1008438-9427-4355-83c6-e1084301e7dd","Type":"ContainerDied","Data":"baf9d096dbc38752fbc488dc8d42bfb420c00073043fb7a3c6040479724d8b4b"} Jul 14 00:54:24.096816 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.096786 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:24.127367 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.127342 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:24.195540 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.195513 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-bundle\") pod \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " Jul 14 00:54:24.195650 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.195555 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-bundle\") pod \"257eede3-e32d-4320-bf9d-b1b2f95b118d\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " Jul 14 00:54:24.195650 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.195584 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m664p\" (UniqueName: \"kubernetes.io/projected/257eede3-e32d-4320-bf9d-b1b2f95b118d-kube-api-access-m664p\") pod \"257eede3-e32d-4320-bf9d-b1b2f95b118d\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " Jul 14 00:54:24.195650 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.195635 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-util\") pod \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " Jul 14 00:54:24.195770 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.195660 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgjnl\" (UniqueName: \"kubernetes.io/projected/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-kube-api-access-jgjnl\") pod \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\" (UID: \"db1c7be4-2ea5-4cb0-9dd0-e440bae11050\") " Jul 14 00:54:24.195770 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.195700 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-util\") pod \"257eede3-e32d-4320-bf9d-b1b2f95b118d\" (UID: \"257eede3-e32d-4320-bf9d-b1b2f95b118d\") " Jul 14 00:54:24.196073 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.196049 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-bundle" (OuterVolumeSpecName: "bundle") pod "db1c7be4-2ea5-4cb0-9dd0-e440bae11050" (UID: "db1c7be4-2ea5-4cb0-9dd0-e440bae11050"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:54:24.196187 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.196117 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-bundle" (OuterVolumeSpecName: "bundle") pod "257eede3-e32d-4320-bf9d-b1b2f95b118d" (UID: "257eede3-e32d-4320-bf9d-b1b2f95b118d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:54:24.197625 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.197597 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/257eede3-e32d-4320-bf9d-b1b2f95b118d-kube-api-access-m664p" (OuterVolumeSpecName: "kube-api-access-m664p") pod "257eede3-e32d-4320-bf9d-b1b2f95b118d" (UID: "257eede3-e32d-4320-bf9d-b1b2f95b118d"). InnerVolumeSpecName "kube-api-access-m664p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:54:24.197720 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.197696 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-kube-api-access-jgjnl" (OuterVolumeSpecName: "kube-api-access-jgjnl") pod "db1c7be4-2ea5-4cb0-9dd0-e440bae11050" (UID: "db1c7be4-2ea5-4cb0-9dd0-e440bae11050"). InnerVolumeSpecName "kube-api-access-jgjnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:54:24.205175 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.205136 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-util" (OuterVolumeSpecName: "util") pod "db1c7be4-2ea5-4cb0-9dd0-e440bae11050" (UID: "db1c7be4-2ea5-4cb0-9dd0-e440bae11050"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:54:24.205447 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.205428 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-util" (OuterVolumeSpecName: "util") pod "257eede3-e32d-4320-bf9d-b1b2f95b118d" (UID: "257eede3-e32d-4320-bf9d-b1b2f95b118d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:54:24.296427 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.296402 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-bundle\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:24.296427 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.296423 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-bundle\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:24.296551 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.296432 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m664p\" (UniqueName: \"kubernetes.io/projected/257eede3-e32d-4320-bf9d-b1b2f95b118d-kube-api-access-m664p\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:24.296551 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.296442 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-util\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:24.296551 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.296452 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgjnl\" (UniqueName: \"kubernetes.io/projected/db1c7be4-2ea5-4cb0-9dd0-e440bae11050-kube-api-access-jgjnl\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:24.296551 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.296459 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/257eede3-e32d-4320-bf9d-b1b2f95b118d-util\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:24.968197 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.968169 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" Jul 14 00:54:24.968197 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.968177 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php" event={"ID":"db1c7be4-2ea5-4cb0-9dd0-e440bae11050","Type":"ContainerDied","Data":"2cf6e0a2b47b689ed36fae2a1c0392882cec113555d7654373f19870efa194fc"} Jul 14 00:54:24.968658 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.968223 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2cf6e0a2b47b689ed36fae2a1c0392882cec113555d7654373f19870efa194fc" Jul 14 00:54:24.970277 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.970244 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" event={"ID":"257eede3-e32d-4320-bf9d-b1b2f95b118d","Type":"ContainerDied","Data":"d6e58b120edeaf1057907abd6bf8ff5b8ee5cb33f70493e3fb15a99ec243e635"} Jul 14 00:54:24.970277 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.970271 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz" Jul 14 00:54:24.970445 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:24.970296 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d6e58b120edeaf1057907abd6bf8ff5b8ee5cb33f70493e3fb15a99ec243e635" Jul 14 00:54:25.138247 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.138228 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:25.140324 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.140283 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:25.202858 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.202833 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-bundle\") pod \"b1008438-9427-4355-83c6-e1084301e7dd\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " Jul 14 00:54:25.202949 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.202864 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-util\") pod \"b1008438-9427-4355-83c6-e1084301e7dd\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " Jul 14 00:54:25.202949 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.202899 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxpn9\" (UniqueName: \"kubernetes.io/projected/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-kube-api-access-sxpn9\") pod \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " Jul 14 00:54:25.202949 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.202927 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-bundle\") pod \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " Jul 14 00:54:25.203054 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.203030 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-util\") pod \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\" (UID: \"55f5baad-9e1e-478f-8ba1-4d1dc53a8527\") " Jul 14 00:54:25.203093 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.203072 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzx8n\" (UniqueName: \"kubernetes.io/projected/b1008438-9427-4355-83c6-e1084301e7dd-kube-api-access-zzx8n\") pod \"b1008438-9427-4355-83c6-e1084301e7dd\" (UID: \"b1008438-9427-4355-83c6-e1084301e7dd\") " Jul 14 00:54:25.203529 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.203505 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-bundle" (OuterVolumeSpecName: "bundle") pod "b1008438-9427-4355-83c6-e1084301e7dd" (UID: "b1008438-9427-4355-83c6-e1084301e7dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:54:25.203658 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.203635 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-bundle" (OuterVolumeSpecName: "bundle") pod "55f5baad-9e1e-478f-8ba1-4d1dc53a8527" (UID: "55f5baad-9e1e-478f-8ba1-4d1dc53a8527"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:54:25.205552 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.205527 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1008438-9427-4355-83c6-e1084301e7dd-kube-api-access-zzx8n" (OuterVolumeSpecName: "kube-api-access-zzx8n") pod "b1008438-9427-4355-83c6-e1084301e7dd" (UID: "b1008438-9427-4355-83c6-e1084301e7dd"). InnerVolumeSpecName "kube-api-access-zzx8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:54:25.205648 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.205560 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-kube-api-access-sxpn9" (OuterVolumeSpecName: "kube-api-access-sxpn9") pod "55f5baad-9e1e-478f-8ba1-4d1dc53a8527" (UID: "55f5baad-9e1e-478f-8ba1-4d1dc53a8527"). InnerVolumeSpecName "kube-api-access-sxpn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:54:25.208760 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.208739 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-util" (OuterVolumeSpecName: "util") pod "b1008438-9427-4355-83c6-e1084301e7dd" (UID: "b1008438-9427-4355-83c6-e1084301e7dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:54:25.208891 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.208873 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-util" (OuterVolumeSpecName: "util") pod "55f5baad-9e1e-478f-8ba1-4d1dc53a8527" (UID: "55f5baad-9e1e-478f-8ba1-4d1dc53a8527"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:54:25.303633 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.303612 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-util\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:25.303708 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.303634 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzx8n\" (UniqueName: \"kubernetes.io/projected/b1008438-9427-4355-83c6-e1084301e7dd-kube-api-access-zzx8n\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:25.303708 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.303645 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-bundle\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:25.303708 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.303653 2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1008438-9427-4355-83c6-e1084301e7dd-util\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:25.303708 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.303662 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxpn9\" (UniqueName: \"kubernetes.io/projected/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-kube-api-access-sxpn9\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:25.303708 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.303671 2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55f5baad-9e1e-478f-8ba1-4d1dc53a8527-bundle\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:54:25.977402 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.977331 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" Jul 14 00:54:25.977402 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.977368 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj" event={"ID":"b1008438-9427-4355-83c6-e1084301e7dd","Type":"ContainerDied","Data":"83cd458025269229cadf2982ceaab23aa7f49d1692e2daa319392d2b9aab1e58"} Jul 14 00:54:25.977861 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.977415 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83cd458025269229cadf2982ceaab23aa7f49d1692e2daa319392d2b9aab1e58" Jul 14 00:54:25.979792 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.979757 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" event={"ID":"55f5baad-9e1e-478f-8ba1-4d1dc53a8527","Type":"ContainerDied","Data":"dcfcdd1e2093c2e9f591122a35b48d9eb8f06417e233f6342aa5da3070847114"} Jul 14 00:54:25.979893 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.979798 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcfcdd1e2093c2e9f591122a35b48d9eb8f06417e233f6342aa5da3070847114" Jul 14 00:54:25.979893 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:25.979872 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb" Jul 14 00:54:32.840907 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:32.840874 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:35.563650 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.563618 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p"] Jul 14 00:54:35.563988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.563930 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="257eede3-e32d-4320-bf9d-b1b2f95b118d" containerName="util" Jul 14 00:54:35.563988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.563941 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="257eede3-e32d-4320-bf9d-b1b2f95b118d" containerName="util" Jul 14 00:54:35.563988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.563953 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1c7be4-2ea5-4cb0-9dd0-e440bae11050" containerName="util" Jul 14 00:54:35.563988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.563959 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1c7be4-2ea5-4cb0-9dd0-e440bae11050" containerName="util" Jul 14 00:54:35.563988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.563968 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1008438-9427-4355-83c6-e1084301e7dd" containerName="util" Jul 14 00:54:35.563988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.563973 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1008438-9427-4355-83c6-e1084301e7dd" containerName="util" Jul 14 00:54:35.563988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.563981 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1008438-9427-4355-83c6-e1084301e7dd" containerName="extract" Jul 14 00:54:35.563988 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.563986 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1008438-9427-4355-83c6-e1084301e7dd" containerName="extract" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.563995 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1c7be4-2ea5-4cb0-9dd0-e440bae11050" containerName="extract" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564001 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1c7be4-2ea5-4cb0-9dd0-e440bae11050" containerName="extract" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564010 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db1c7be4-2ea5-4cb0-9dd0-e440bae11050" containerName="pull" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564015 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="db1c7be4-2ea5-4cb0-9dd0-e440bae11050" containerName="pull" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564021 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="257eede3-e32d-4320-bf9d-b1b2f95b118d" containerName="pull" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564025 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="257eede3-e32d-4320-bf9d-b1b2f95b118d" containerName="pull" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564031 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1008438-9427-4355-83c6-e1084301e7dd" containerName="pull" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564035 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1008438-9427-4355-83c6-e1084301e7dd" containerName="pull" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564042 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55f5baad-9e1e-478f-8ba1-4d1dc53a8527" containerName="util" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564046 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="55f5baad-9e1e-478f-8ba1-4d1dc53a8527" containerName="util" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564052 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55f5baad-9e1e-478f-8ba1-4d1dc53a8527" containerName="extract" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564057 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="55f5baad-9e1e-478f-8ba1-4d1dc53a8527" containerName="extract" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564062 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="257eede3-e32d-4320-bf9d-b1b2f95b118d" containerName="extract" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564068 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="257eede3-e32d-4320-bf9d-b1b2f95b118d" containerName="extract" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564074 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55f5baad-9e1e-478f-8ba1-4d1dc53a8527" containerName="pull" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564079 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="55f5baad-9e1e-478f-8ba1-4d1dc53a8527" containerName="pull" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564128 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="55f5baad-9e1e-478f-8ba1-4d1dc53a8527" containerName="extract" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564135 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="db1c7be4-2ea5-4cb0-9dd0-e440bae11050" containerName="extract" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564140 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="257eede3-e32d-4320-bf9d-b1b2f95b118d" containerName="extract" Jul 14 00:54:35.564261 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.564160 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1008438-9427-4355-83c6-e1084301e7dd" containerName="extract" Jul 14 00:54:35.567082 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.567065 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:54:35.572353 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.572331 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-287cf\"" Jul 14 00:54:35.590958 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.590939 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p"] Jul 14 00:54:35.675610 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.675591 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqnlr\" (UniqueName: \"kubernetes.io/projected/a34562ef-787a-4ff8-a842-18823bbd674c-kube-api-access-zqnlr\") pod \"limitador-operator-controller-manager-859d7b78bd-4s44p\" (UID: \"a34562ef-787a-4ff8-a842-18823bbd674c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:54:35.776469 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.776443 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqnlr\" (UniqueName: \"kubernetes.io/projected/a34562ef-787a-4ff8-a842-18823bbd674c-kube-api-access-zqnlr\") pod \"limitador-operator-controller-manager-859d7b78bd-4s44p\" (UID: \"a34562ef-787a-4ff8-a842-18823bbd674c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:54:35.785716 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.785690 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqnlr\" (UniqueName: \"kubernetes.io/projected/a34562ef-787a-4ff8-a842-18823bbd674c-kube-api-access-zqnlr\") pod \"limitador-operator-controller-manager-859d7b78bd-4s44p\" (UID: \"a34562ef-787a-4ff8-a842-18823bbd674c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:54:35.876892 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:35.876840 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:54:36.021101 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:36.021078 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p"] Jul 14 00:54:36.023342 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:54:36.023311 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34562ef_787a_4ff8_a842_18823bbd674c.slice/crio-7c333430458027f9c1c6b6649bd565d6a3bb66ce8021059894690558c11298c4 WatchSource:0}: Error finding container 7c333430458027f9c1c6b6649bd565d6a3bb66ce8021059894690558c11298c4: Status 404 returned error can't find the container with id 7c333430458027f9c1c6b6649bd565d6a3bb66ce8021059894690558c11298c4 Jul 14 00:54:36.030226 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:36.030200 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" event={"ID":"a34562ef-787a-4ff8-a842-18823bbd674c","Type":"ContainerStarted","Data":"7c333430458027f9c1c6b6649bd565d6a3bb66ce8021059894690558c11298c4"} Jul 14 00:54:36.270454 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:36.270388 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:36.835138 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:36.835102 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:36.995650 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:36.995322 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-595f89b4d9-rvrbn"] Jul 14 00:54:36.999232 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:36.999211 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.001702 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.001580 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 00:54:37.001702 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.001645 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 00:54:37.001922 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.001798 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4t8jn\"" Jul 14 00:54:37.001922 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.001905 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 00:54:37.002048 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.001941 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 00:54:37.002048 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.001967 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 00:54:37.007280 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.007260 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 00:54:37.012701 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.012680 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-595f89b4d9-rvrbn"] Jul 14 00:54:37.088734 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.088674 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-oauth-serving-cert\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.088734 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.088708 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-service-ca\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.088889 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.088745 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01e70466-94a2-429f-b650-e59133fa738d-console-serving-cert\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.088889 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.088804 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-trusted-ca-bundle\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.088889 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.088840 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw2v5\" (UniqueName: \"kubernetes.io/projected/01e70466-94a2-429f-b650-e59133fa738d-kube-api-access-kw2v5\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.088889 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.088862 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01e70466-94a2-429f-b650-e59133fa738d-console-oauth-config\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.089282 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.088891 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-console-config\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.092452 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.092428 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:37.189689 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.189663 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01e70466-94a2-429f-b650-e59133fa738d-console-oauth-config\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.189832 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.189700 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-console-config\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.189832 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.189725 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-oauth-serving-cert\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.189832 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.189743 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-service-ca\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.189832 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.189782 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01e70466-94a2-429f-b650-e59133fa738d-console-serving-cert\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.190059 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.189847 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-trusted-ca-bundle\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.190059 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.189884 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw2v5\" (UniqueName: \"kubernetes.io/projected/01e70466-94a2-429f-b650-e59133fa738d-kube-api-access-kw2v5\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.190721 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.190691 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-console-config\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.190845 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.190754 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-service-ca\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.190845 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.190759 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-oauth-serving-cert\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.190972 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.190926 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01e70466-94a2-429f-b650-e59133fa738d-trusted-ca-bundle\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.192560 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.192519 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/01e70466-94a2-429f-b650-e59133fa738d-console-oauth-config\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.192654 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.192643 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/01e70466-94a2-429f-b650-e59133fa738d-console-serving-cert\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.198123 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.198100 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw2v5\" (UniqueName: \"kubernetes.io/projected/01e70466-94a2-429f-b650-e59133fa738d-kube-api-access-kw2v5\") pod \"console-595f89b4d9-rvrbn\" (UID: \"01e70466-94a2-429f-b650-e59133fa738d\") " pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.261374 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.261353 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:37.312764 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.312742 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:37.397235 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.397201 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:37.460241 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.460218 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-595f89b4d9-rvrbn"] Jul 14 00:54:37.462125 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:54:37.462099 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e70466_94a2_429f_b650_e59133fa738d.slice/crio-cc2cdef2589bceb5fd268f1837327263f5e7a8bc57dfd0f7affc9ee3cc1127e5 WatchSource:0}: Error finding container cc2cdef2589bceb5fd268f1837327263f5e7a8bc57dfd0f7affc9ee3cc1127e5: Status 404 returned error can't find the container with id cc2cdef2589bceb5fd268f1837327263f5e7a8bc57dfd0f7affc9ee3cc1127e5 Jul 14 00:54:37.779883 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.779792 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:37.883137 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:37.883103 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:38.044616 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:38.044592 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" event={"ID":"a34562ef-787a-4ff8-a842-18823bbd674c","Type":"ContainerStarted","Data":"a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6"} Jul 14 00:54:38.044848 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:38.044643 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:54:38.046120 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:38.046100 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595f89b4d9-rvrbn" event={"ID":"01e70466-94a2-429f-b650-e59133fa738d","Type":"ContainerStarted","Data":"5264686cb4a0c05b5366c2eddea41f7ffe1ca4d76a73a0f3e799e80850c8be84"} Jul 14 00:54:38.046225 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:38.046137 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-595f89b4d9-rvrbn" event={"ID":"01e70466-94a2-429f-b650-e59133fa738d","Type":"ContainerStarted","Data":"cc2cdef2589bceb5fd268f1837327263f5e7a8bc57dfd0f7affc9ee3cc1127e5"} Jul 14 00:54:38.060133 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:38.060092 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" podStartSLOduration=1.197044729 podStartE2EDuration="3.060077461s" totalImagesPullingTime="1.863032732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:54:38.058313082 +0000 UTC m=+614.909680451" watchObservedRunningTime="2026-07-14 00:54:38.060077461 +0000 UTC m=+614.911444830" Jul 14 00:54:38.075140 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:38.075103 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-595f89b4d9-rvrbn" podStartSLOduration=2.075090243 podStartE2EDuration="2.075090243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:54:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:54:38.074139078 +0000 UTC m=+614.925506448" watchObservedRunningTime="2026-07-14 00:54:38.075090243 +0000 UTC m=+614.926457613" Jul 14 00:54:39.328765 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:39.328724 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g7mht"] Jul 14 00:54:39.332619 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:39.332597 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g7mht" Jul 14 00:54:39.334838 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:39.334811 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-n6jnh\"" Jul 14 00:54:39.343555 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:39.343517 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g7mht"] Jul 14 00:54:39.406790 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:39.406705 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nmwf\" (UniqueName: \"kubernetes.io/projected/b6e80bb6-828a-4dda-9a70-0bda2c7ac0b3-kube-api-access-7nmwf\") pod \"authorino-operator-75cc8d5cb7-g7mht\" (UID: \"b6e80bb6-828a-4dda-9a70-0bda2c7ac0b3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g7mht" Jul 14 00:54:39.508172 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:39.508122 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nmwf\" (UniqueName: \"kubernetes.io/projected/b6e80bb6-828a-4dda-9a70-0bda2c7ac0b3-kube-api-access-7nmwf\") pod \"authorino-operator-75cc8d5cb7-g7mht\" (UID: \"b6e80bb6-828a-4dda-9a70-0bda2c7ac0b3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g7mht" Jul 14 00:54:39.517747 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:39.517721 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nmwf\" (UniqueName: \"kubernetes.io/projected/b6e80bb6-828a-4dda-9a70-0bda2c7ac0b3-kube-api-access-7nmwf\") pod \"authorino-operator-75cc8d5cb7-g7mht\" (UID: \"b6e80bb6-828a-4dda-9a70-0bda2c7ac0b3\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-g7mht" Jul 14 00:54:39.645485 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:39.645423 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g7mht" Jul 14 00:54:39.774431 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:39.774371 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-g7mht"] Jul 14 00:54:39.776639 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:54:39.776610 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e80bb6_828a_4dda_9a70_0bda2c7ac0b3.slice/crio-4e600c916797ac892396206729b479ba273052e856ee5ef3994f60ffeeb41d58 WatchSource:0}: Error finding container 4e600c916797ac892396206729b479ba273052e856ee5ef3994f60ffeeb41d58: Status 404 returned error can't find the container with id 4e600c916797ac892396206729b479ba273052e856ee5ef3994f60ffeeb41d58 Jul 14 00:54:39.968480 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:39.968420 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:40.051223 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:40.051191 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:40.056360 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:40.056331 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g7mht" event={"ID":"b6e80bb6-828a-4dda-9a70-0bda2c7ac0b3","Type":"ContainerStarted","Data":"4e600c916797ac892396206729b479ba273052e856ee5ef3994f60ffeeb41d58"} Jul 14 00:54:42.495011 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:42.494935 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:43.075688 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:43.075651 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g7mht" event={"ID":"b6e80bb6-828a-4dda-9a70-0bda2c7ac0b3","Type":"ContainerStarted","Data":"5ae4ed216d83cb15d4cc45a4273a680de0456f3e6c1793280e68c4ec8ab56f84"} Jul 14 00:54:43.075848 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:43.075831 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g7mht" Jul 14 00:54:43.090786 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:43.090744 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g7mht" podStartSLOduration=1.363313096 podStartE2EDuration="4.090728095s" totalImagesPullingTime="2.727414999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:54:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:54:43.088923508 +0000 UTC m=+619.940290878" watchObservedRunningTime="2026-07-14 00:54:43.090728095 +0000 UTC m=+619.942095522" Jul 14 00:54:44.293536 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.293500 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:44.301773 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.301735 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w"] Jul 14 00:54:44.304349 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.304327 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:54:44.306366 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.306348 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nxppt\"" Jul 14 00:54:44.315916 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.315895 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w"] Jul 14 00:54:44.345823 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.345799 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff9f4917-9f80-4c7a-baaf-2739d27d9610-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" (UID: \"ff9f4917-9f80-4c7a-baaf-2739d27d9610\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:54:44.345933 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.345831 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgjr8\" (UniqueName: \"kubernetes.io/projected/ff9f4917-9f80-4c7a-baaf-2739d27d9610-kube-api-access-dgjr8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" (UID: \"ff9f4917-9f80-4c7a-baaf-2739d27d9610\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:54:44.447282 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.447255 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff9f4917-9f80-4c7a-baaf-2739d27d9610-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" (UID: \"ff9f4917-9f80-4c7a-baaf-2739d27d9610\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:54:44.447376 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.447289 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgjr8\" (UniqueName: \"kubernetes.io/projected/ff9f4917-9f80-4c7a-baaf-2739d27d9610-kube-api-access-dgjr8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" (UID: \"ff9f4917-9f80-4c7a-baaf-2739d27d9610\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:54:44.447613 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.447595 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff9f4917-9f80-4c7a-baaf-2739d27d9610-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" (UID: \"ff9f4917-9f80-4c7a-baaf-2739d27d9610\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:54:44.462658 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.462631 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgjr8\" (UniqueName: \"kubernetes.io/projected/ff9f4917-9f80-4c7a-baaf-2739d27d9610-kube-api-access-dgjr8\") pod \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" (UID: \"ff9f4917-9f80-4c7a-baaf-2739d27d9610\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:54:44.614648 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.614592 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:54:44.749805 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:44.749779 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w"] Jul 14 00:54:44.752325 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:54:44.752302 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff9f4917_9f80_4c7a_baaf_2739d27d9610.slice/crio-563bda8d3d7e30d3083ca8649ef5c54e25a046ddbca7476c92693db7ffaf25e0 WatchSource:0}: Error finding container 563bda8d3d7e30d3083ca8649ef5c54e25a046ddbca7476c92693db7ffaf25e0: Status 404 returned error can't find the container with id 563bda8d3d7e30d3083ca8649ef5c54e25a046ddbca7476c92693db7ffaf25e0 Jul 14 00:54:45.086490 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:45.086460 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" event={"ID":"ff9f4917-9f80-4c7a-baaf-2739d27d9610","Type":"ContainerStarted","Data":"563bda8d3d7e30d3083ca8649ef5c54e25a046ddbca7476c92693db7ffaf25e0"} Jul 14 00:54:45.448432 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:45.448342 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:47.313595 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:47.313562 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:47.314056 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:47.313628 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:47.318679 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:47.318650 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:47.482795 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:47.482756 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:48.104191 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:48.104162 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-595f89b4d9-rvrbn" Jul 14 00:54:48.272999 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:48.272960 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:48.468823 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:48.468744 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:48.566056 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:48.566027 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:49.052766 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:49.052740 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:54:49.058969 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:49.058949 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:49.164837 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:49.164811 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:52.125073 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:52.125037 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" event={"ID":"ff9f4917-9f80-4c7a-baaf-2739d27d9610","Type":"ContainerStarted","Data":"412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48"} Jul 14 00:54:52.125520 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:52.125118 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:54:52.143125 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:52.143078 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" podStartSLOduration=1.146746134 podStartE2EDuration="8.14306498s" totalImagesPullingTime="6.996318846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:54:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:54:52.141272714 +0000 UTC m=+628.992640084" watchObservedRunningTime="2026-07-14 00:54:52.14306498 +0000 UTC m=+628.994432349" Jul 14 00:54:53.037608 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:53.037565 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:53.135949 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:53.135921 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:53.445081 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:53.445000 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:53.543999 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:53.543972 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:54.082961 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:54.082937 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-g7mht" Jul 14 00:54:54.365626 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:54.365539 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:54.493630 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:54.493593 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:57.432044 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:57.432007 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:54:57.532551 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:54:57.532520 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:03.133212 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:03.133185 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:03.967099 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:03.967063 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:04.070755 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.070723 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:04.825218 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.825188 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w"] Jul 14 00:55:04.825609 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.825375 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" containerName="manager" containerID="cri-o://412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48" gracePeriod=2 Jul 14 00:55:04.835277 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.835251 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w"] Jul 14 00:55:04.837680 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:04.837653 2557 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ff9f4917-9f80-4c7a-baaf-2739d27d9610\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\": pods \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:04.850570 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.850547 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj"] Jul 14 00:55:04.850918 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.850904 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" containerName="manager" Jul 14 00:55:04.850965 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.850920 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" containerName="manager" Jul 14 00:55:04.850999 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.850983 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" containerName="manager" Jul 14 00:55:04.854045 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.854024 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p"] Jul 14 00:55:04.854215 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.854191 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:04.854215 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.854200 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" containerName="manager" containerID="cri-o://a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6" gracePeriod=2 Jul 14 00:55:04.856402 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:04.856370 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:04.867334 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.867307 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p"] Jul 14 00:55:04.870868 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:04.870833 2557 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"a34562ef-787a-4ff8-a842-18823bbd674c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-4s44p\": pods \"limitador-operator-controller-manager-859d7b78bd-4s44p\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:55:04.873645 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:04.873618 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4s44p\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:55:04.874286 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.874266 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj"] Jul 14 00:55:04.887520 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.887495 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5"] Jul 14 00:55:04.887897 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.887881 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" containerName="manager" Jul 14 00:55:04.888006 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.887900 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" containerName="manager" Jul 14 00:55:04.888006 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.887996 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" containerName="manager" Jul 14 00:55:04.891070 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.891049 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5" Jul 14 00:55:04.904045 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:04.904013 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:04.904250 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:04.904230 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5"] Jul 14 00:55:04.905628 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:04.905595 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:04.923236 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:04.923207 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4s44p\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:55:05.008529 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.008488 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12e884f4-e5cf-41b9-8563-ef492eb55121-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69wdj\" (UID: \"12e884f4-e5cf-41b9-8563-ef492eb55121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:05.008529 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.008529 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z9kt\" (UniqueName: \"kubernetes.io/projected/28802c53-86c9-4c60-94ec-66daf134046c-kube-api-access-8z9kt\") pod \"limitador-operator-controller-manager-859d7b78bd-cvgq5\" (UID: \"28802c53-86c9-4c60-94ec-66daf134046c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5" Jul 14 00:55:05.008763 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.008594 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wwkf\" (UniqueName: \"kubernetes.io/projected/12e884f4-e5cf-41b9-8563-ef492eb55121-kube-api-access-2wwkf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69wdj\" (UID: \"12e884f4-e5cf-41b9-8563-ef492eb55121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:05.090859 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.090836 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:05.092412 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.092390 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:05.093270 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.093256 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:55:05.093533 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.093515 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4s44p\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:55:05.094750 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.094733 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4s44p\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:55:05.095741 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.095724 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:05.109133 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.109111 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wwkf\" (UniqueName: \"kubernetes.io/projected/12e884f4-e5cf-41b9-8563-ef492eb55121-kube-api-access-2wwkf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69wdj\" (UID: \"12e884f4-e5cf-41b9-8563-ef492eb55121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:05.109240 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.109174 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12e884f4-e5cf-41b9-8563-ef492eb55121-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69wdj\" (UID: \"12e884f4-e5cf-41b9-8563-ef492eb55121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:05.109240 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.109198 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z9kt\" (UniqueName: \"kubernetes.io/projected/28802c53-86c9-4c60-94ec-66daf134046c-kube-api-access-8z9kt\") pod \"limitador-operator-controller-manager-859d7b78bd-cvgq5\" (UID: \"28802c53-86c9-4c60-94ec-66daf134046c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5" Jul 14 00:55:05.109539 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.109519 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12e884f4-e5cf-41b9-8563-ef492eb55121-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69wdj\" (UID: \"12e884f4-e5cf-41b9-8563-ef492eb55121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:05.118918 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.118896 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z9kt\" (UniqueName: \"kubernetes.io/projected/28802c53-86c9-4c60-94ec-66daf134046c-kube-api-access-8z9kt\") pod \"limitador-operator-controller-manager-859d7b78bd-cvgq5\" (UID: \"28802c53-86c9-4c60-94ec-66daf134046c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5" Jul 14 00:55:05.119035 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.119019 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wwkf\" (UniqueName: \"kubernetes.io/projected/12e884f4-e5cf-41b9-8563-ef492eb55121-kube-api-access-2wwkf\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69wdj\" (UID: \"12e884f4-e5cf-41b9-8563-ef492eb55121\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:05.191025 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.191002 2557 generic.go:356] "Generic (PLEG): container finished" podID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" containerID="412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48" exitCode=0 Jul 14 00:55:05.191114 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.191045 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:05.191114 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.191064 2557 scope.go:117] "RemoveContainer" containerID="412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48" Jul 14 00:55:05.192534 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.192514 2557 generic.go:356] "Generic (PLEG): container finished" podID="a34562ef-787a-4ff8-a842-18823bbd674c" containerID="a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6" exitCode=0 Jul 14 00:55:05.192631 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.192551 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:55:05.192795 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.192772 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:05.194040 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.194020 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4s44p\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:55:05.195297 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.195268 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:05.196509 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.196488 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4s44p\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:55:05.199232 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.199201 2557 scope.go:117] "RemoveContainer" containerID="412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48" Jul 14 00:55:05.199455 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.199439 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48\": container with ID starting with 412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48 not found: ID does not exist" containerID="412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48" Jul 14 00:55:05.199500 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.199460 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48"} err="failed to get container status \"412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48\": rpc error: code = NotFound desc = could not find container \"412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48\": container with ID starting with 412a832c60eb2623c13a997b8261c6cf6225277a7f10009f2db3f4712e5a1b48 not found: ID does not exist" Jul 14 00:55:05.199500 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.199474 2557 scope.go:117] "RemoveContainer" containerID="a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6" Jul 14 00:55:05.207060 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.207043 2557 scope.go:117] "RemoveContainer" containerID="a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6" Jul 14 00:55:05.207305 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.207290 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6\": container with ID starting with a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6 not found: ID does not exist" containerID="a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6" Jul 14 00:55:05.207353 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.207318 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6"} err="failed to get container status \"a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6\": rpc error: code = NotFound desc = could not find container \"a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6\": container with ID starting with a7b316dfe7cc0a691568582b5c89ab8fd0c21aea1e1091b83f7e719bad7a5eb6 not found: ID does not exist" Jul 14 00:55:05.209578 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.209562 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgjr8\" (UniqueName: \"kubernetes.io/projected/ff9f4917-9f80-4c7a-baaf-2739d27d9610-kube-api-access-dgjr8\") pod \"ff9f4917-9f80-4c7a-baaf-2739d27d9610\" (UID: \"ff9f4917-9f80-4c7a-baaf-2739d27d9610\") " Jul 14 00:55:05.209633 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.209625 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff9f4917-9f80-4c7a-baaf-2739d27d9610-extensions-socket-volume\") pod \"ff9f4917-9f80-4c7a-baaf-2739d27d9610\" (UID: \"ff9f4917-9f80-4c7a-baaf-2739d27d9610\") " Jul 14 00:55:05.209693 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.209652 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqnlr\" (UniqueName: \"kubernetes.io/projected/a34562ef-787a-4ff8-a842-18823bbd674c-kube-api-access-zqnlr\") pod \"a34562ef-787a-4ff8-a842-18823bbd674c\" (UID: \"a34562ef-787a-4ff8-a842-18823bbd674c\") " Jul 14 00:55:05.210175 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.210127 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9f4917-9f80-4c7a-baaf-2739d27d9610-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ff9f4917-9f80-4c7a-baaf-2739d27d9610" (UID: "ff9f4917-9f80-4c7a-baaf-2739d27d9610"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:55:05.211468 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.211448 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9f4917-9f80-4c7a-baaf-2739d27d9610-kube-api-access-dgjr8" (OuterVolumeSpecName: "kube-api-access-dgjr8") pod "ff9f4917-9f80-4c7a-baaf-2739d27d9610" (UID: "ff9f4917-9f80-4c7a-baaf-2739d27d9610"). InnerVolumeSpecName "kube-api-access-dgjr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:55:05.211582 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.211565 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a34562ef-787a-4ff8-a842-18823bbd674c-kube-api-access-zqnlr" (OuterVolumeSpecName: "kube-api-access-zqnlr") pod "a34562ef-787a-4ff8-a842-18823bbd674c" (UID: "a34562ef-787a-4ff8-a842-18823bbd674c"). InnerVolumeSpecName "kube-api-access-zqnlr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:55:05.259898 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.259861 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:05.265850 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.265817 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5" Jul 14 00:55:05.310521 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.310485 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ff9f4917-9f80-4c7a-baaf-2739d27d9610-extensions-socket-volume\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:55:05.310521 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.310523 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqnlr\" (UniqueName: \"kubernetes.io/projected/a34562ef-787a-4ff8-a842-18823bbd674c-kube-api-access-zqnlr\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:55:05.310747 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.310537 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgjr8\" (UniqueName: \"kubernetes.io/projected/ff9f4917-9f80-4c7a-baaf-2739d27d9610-kube-api-access-dgjr8\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:55:05.412926 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.412904 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj"] Jul 14 00:55:05.414022 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:55:05.414000 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12e884f4_e5cf_41b9_8563_ef492eb55121.slice/crio-3b29ac38fab4b8cee08bd84b5cf5aaae6085e7708bef26143b7b69fcec127528 WatchSource:0}: Error finding container 3b29ac38fab4b8cee08bd84b5cf5aaae6085e7708bef26143b7b69fcec127528: Status 404 returned error can't find the container with id 3b29ac38fab4b8cee08bd84b5cf5aaae6085e7708bef26143b7b69fcec127528 Jul 14 00:55:05.431221 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.431200 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5"] Jul 14 00:55:05.432491 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:55:05.432462 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28802c53_86c9_4c60_94ec_66daf134046c.slice/crio-fc1aaa4dc5c37df66e4ff0fe29d5da24323c33711e680cb223a25a556107598b WatchSource:0}: Error finding container fc1aaa4dc5c37df66e4ff0fe29d5da24323c33711e680cb223a25a556107598b: Status 404 returned error can't find the container with id fc1aaa4dc5c37df66e4ff0fe29d5da24323c33711e680cb223a25a556107598b Jul 14 00:55:05.503468 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.503441 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4s44p\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:55:05.504796 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.504772 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:05.506079 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.506061 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-vsh7w\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-vsh7w" Jul 14 00:55:05.507171 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:05.507133 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4s44p\" is forbidden: User \"system:node:ip-10-0-134-143.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-134-143.ec2.internal' and this object" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4s44p" Jul 14 00:55:05.683520 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.683438 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a34562ef-787a-4ff8-a842-18823bbd674c" path="/var/lib/kubelet/pods/a34562ef-787a-4ff8-a842-18823bbd674c/volumes" Jul 14 00:55:05.683850 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:05.683833 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff9f4917-9f80-4c7a-baaf-2739d27d9610" path="/var/lib/kubelet/pods/ff9f4917-9f80-4c7a-baaf-2739d27d9610/volumes" Jul 14 00:55:06.200362 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:06.200321 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5" event={"ID":"28802c53-86c9-4c60-94ec-66daf134046c","Type":"ContainerStarted","Data":"7a083a1c1a71d6f86331655466b7bed105ea033f8877048d68ff9b33dc13a81c"} Jul 14 00:55:06.200807 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:06.200378 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5" event={"ID":"28802c53-86c9-4c60-94ec-66daf134046c","Type":"ContainerStarted","Data":"fc1aaa4dc5c37df66e4ff0fe29d5da24323c33711e680cb223a25a556107598b"} Jul 14 00:55:06.200807 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:06.200429 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5" Jul 14 00:55:06.202786 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:06.202757 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" event={"ID":"12e884f4-e5cf-41b9-8563-ef492eb55121","Type":"ContainerStarted","Data":"85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64"} Jul 14 00:55:06.202908 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:06.202801 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:06.202908 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:06.202816 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" event={"ID":"12e884f4-e5cf-41b9-8563-ef492eb55121","Type":"ContainerStarted","Data":"3b29ac38fab4b8cee08bd84b5cf5aaae6085e7708bef26143b7b69fcec127528"} Jul 14 00:55:06.217347 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:06.217307 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5" podStartSLOduration=2.217294748 podStartE2EDuration="2.217294748s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:55:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:55:06.215849628 +0000 UTC m=+643.067216998" watchObservedRunningTime="2026-07-14 00:55:06.217294748 +0000 UTC m=+643.068662117" Jul 14 00:55:06.236931 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:06.236886 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" podStartSLOduration=2.236874844 podStartE2EDuration="2.236874844s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:55:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:55:06.236092111 +0000 UTC m=+643.087459480" watchObservedRunningTime="2026-07-14 00:55:06.236874844 +0000 UTC m=+643.088242208" Jul 14 00:55:08.345871 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:08.345836 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:08.445949 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:08.445920 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:12.080420 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:12.080388 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:12.180426 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:12.180396 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:12.280811 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:12.280778 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:12.336456 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:12.336362 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:17.211775 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:17.211740 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:17.212220 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:17.211912 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cvgq5" Jul 14 00:55:17.295661 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:17.295606 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:17.397989 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:17.397958 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:21.530036 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.530002 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj"] Jul 14 00:55:21.530440 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.530196 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" podUID="12e884f4-e5cf-41b9-8563-ef492eb55121" containerName="manager" containerID="cri-o://85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64" gracePeriod=10 Jul 14 00:55:21.734300 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.734275 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t"] Jul 14 00:55:21.737604 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.737583 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 00:55:21.752929 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.752904 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t"] Jul 14 00:55:21.793180 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.793144 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:21.846184 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.846140 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mjdf\" (UniqueName: \"kubernetes.io/projected/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-kube-api-access-2mjdf\") pod \"kuadrant-operator-controller-manager-86878c769f-rj78t\" (UID: \"36d2e455-dbc5-4663-b70b-f1e8ec0dde97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 00:55:21.846348 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.846204 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rj78t\" (UID: \"36d2e455-dbc5-4663-b70b-f1e8ec0dde97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 00:55:21.947314 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.947282 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wwkf\" (UniqueName: \"kubernetes.io/projected/12e884f4-e5cf-41b9-8563-ef492eb55121-kube-api-access-2wwkf\") pod \"12e884f4-e5cf-41b9-8563-ef492eb55121\" (UID: \"12e884f4-e5cf-41b9-8563-ef492eb55121\") " Jul 14 00:55:21.947472 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.947333 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12e884f4-e5cf-41b9-8563-ef492eb55121-extensions-socket-volume\") pod \"12e884f4-e5cf-41b9-8563-ef492eb55121\" (UID: \"12e884f4-e5cf-41b9-8563-ef492eb55121\") " Jul 14 00:55:21.947515 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.947486 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mjdf\" (UniqueName: \"kubernetes.io/projected/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-kube-api-access-2mjdf\") pod \"kuadrant-operator-controller-manager-86878c769f-rj78t\" (UID: \"36d2e455-dbc5-4663-b70b-f1e8ec0dde97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 00:55:21.947551 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.947517 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rj78t\" (UID: \"36d2e455-dbc5-4663-b70b-f1e8ec0dde97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 00:55:21.947707 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.947685 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12e884f4-e5cf-41b9-8563-ef492eb55121-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "12e884f4-e5cf-41b9-8563-ef492eb55121" (UID: "12e884f4-e5cf-41b9-8563-ef492eb55121"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 00:55:21.947961 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.947939 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rj78t\" (UID: \"36d2e455-dbc5-4663-b70b-f1e8ec0dde97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 00:55:21.949335 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.949316 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12e884f4-e5cf-41b9-8563-ef492eb55121-kube-api-access-2wwkf" (OuterVolumeSpecName: "kube-api-access-2wwkf") pod "12e884f4-e5cf-41b9-8563-ef492eb55121" (UID: "12e884f4-e5cf-41b9-8563-ef492eb55121"). InnerVolumeSpecName "kube-api-access-2wwkf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:55:21.961463 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:21.961442 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mjdf\" (UniqueName: \"kubernetes.io/projected/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-kube-api-access-2mjdf\") pod \"kuadrant-operator-controller-manager-86878c769f-rj78t\" (UID: \"36d2e455-dbc5-4663-b70b-f1e8ec0dde97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 00:55:22.048291 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.048208 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 00:55:22.048474 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.048453 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wwkf\" (UniqueName: \"kubernetes.io/projected/12e884f4-e5cf-41b9-8563-ef492eb55121-kube-api-access-2wwkf\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:55:22.048536 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.048480 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/12e884f4-e5cf-41b9-8563-ef492eb55121-extensions-socket-volume\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:55:22.179987 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.179955 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t"] Jul 14 00:55:22.182191 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:55:22.182157 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36d2e455_dbc5_4663_b70b_f1e8ec0dde97.slice/crio-215e993f3fbb3624c83088bd4fdbaab4301f4446a9d0ec9f8f4c91946db22064 WatchSource:0}: Error finding container 215e993f3fbb3624c83088bd4fdbaab4301f4446a9d0ec9f8f4c91946db22064: Status 404 returned error can't find the container with id 215e993f3fbb3624c83088bd4fdbaab4301f4446a9d0ec9f8f4c91946db22064 Jul 14 00:55:22.289168 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.289119 2557 generic.go:356] "Generic (PLEG): container finished" podID="12e884f4-e5cf-41b9-8563-ef492eb55121" containerID="85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64" exitCode=0 Jul 14 00:55:22.289382 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.289181 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" Jul 14 00:55:22.289382 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.289207 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" event={"ID":"12e884f4-e5cf-41b9-8563-ef492eb55121","Type":"ContainerDied","Data":"85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64"} Jul 14 00:55:22.289382 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.289268 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj" event={"ID":"12e884f4-e5cf-41b9-8563-ef492eb55121","Type":"ContainerDied","Data":"3b29ac38fab4b8cee08bd84b5cf5aaae6085e7708bef26143b7b69fcec127528"} Jul 14 00:55:22.289382 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.289298 2557 scope.go:117] "RemoveContainer" containerID="85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64" Jul 14 00:55:22.290891 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.290866 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" event={"ID":"36d2e455-dbc5-4663-b70b-f1e8ec0dde97","Type":"ContainerStarted","Data":"f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795"} Jul 14 00:55:22.291000 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.290916 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 00:55:22.291000 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.290933 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" event={"ID":"36d2e455-dbc5-4663-b70b-f1e8ec0dde97","Type":"ContainerStarted","Data":"215e993f3fbb3624c83088bd4fdbaab4301f4446a9d0ec9f8f4c91946db22064"} Jul 14 00:55:22.297525 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.297501 2557 scope.go:117] "RemoveContainer" containerID="85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64" Jul 14 00:55:22.297751 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:22.297733 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64\": container with ID starting with 85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64 not found: ID does not exist" containerID="85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64" Jul 14 00:55:22.297800 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.297760 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64"} err="failed to get container status \"85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64\": rpc error: code = NotFound desc = could not find container \"85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64\": container with ID starting with 85006395defac247319ff091b849513fb2d7486387d5fdf0c7fd61a5a0a10c64 not found: ID does not exist" Jul 14 00:55:22.307867 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.307800 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" podStartSLOduration=1.3077885949999999 podStartE2EDuration="1.307788595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:55:22.305830851 +0000 UTC m=+659.157198223" watchObservedRunningTime="2026-07-14 00:55:22.307788595 +0000 UTC m=+659.159155966" Jul 14 00:55:22.321842 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.321817 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj"] Jul 14 00:55:22.325534 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:22.325515 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69wdj"] Jul 14 00:55:23.677390 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:23.677364 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12e884f4-e5cf-41b9-8563-ef492eb55121" path="/var/lib/kubelet/pods/12e884f4-e5cf-41b9-8563-ef492eb55121/volumes" Jul 14 00:55:24.020785 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:24.020717 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:24.121189 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:24.121141 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:33.299784 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:33.299752 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 00:55:37.566697 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.566651 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:37.614601 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.614569 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:37.865168 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.865053 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd"] Jul 14 00:55:37.865513 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.865495 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12e884f4-e5cf-41b9-8563-ef492eb55121" containerName="manager" Jul 14 00:55:37.865601 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.865517 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="12e884f4-e5cf-41b9-8563-ef492eb55121" containerName="manager" Jul 14 00:55:37.865654 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.865627 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="12e884f4-e5cf-41b9-8563-ef492eb55121" containerName="manager" Jul 14 00:55:37.870415 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.870393 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.872252 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.872233 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-57ss7\"" Jul 14 00:55:37.874564 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.874543 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad7dd378-440e-4ae4-b524-197073a5b229-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.874690 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.874582 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.878189 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.878131 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad7dd378-440e-4ae4-b524-197073a5b229-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.878401 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.878366 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.878586 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.878560 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gll7z\" (UniqueName: \"kubernetes.io/projected/ad7dd378-440e-4ae4-b524-197073a5b229-kube-api-access-gll7z\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.878778 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.878759 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.879026 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.879008 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.879192 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.879175 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad7dd378-440e-4ae4-b524-197073a5b229-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.882688 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.882659 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.884368 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.884345 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:37.886506 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.886438 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd"] Jul 14 00:55:37.983484 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.983446 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.983484 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.983487 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad7dd378-440e-4ae4-b524-197073a5b229-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.983731 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.983635 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.983731 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.983690 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gll7z\" (UniqueName: \"kubernetes.io/projected/ad7dd378-440e-4ae4-b524-197073a5b229-kube-api-access-gll7z\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.983851 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.983735 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.983851 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.983775 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.983851 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.983800 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad7dd378-440e-4ae4-b524-197073a5b229-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.983851 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.983847 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.984056 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.983931 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad7dd378-440e-4ae4-b524-197073a5b229-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.984056 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.984010 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.984430 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.984080 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.984430 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.984336 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ad7dd378-440e-4ae4-b524-197073a5b229-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.984430 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.984366 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.984582 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.984434 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.986300 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.986278 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ad7dd378-440e-4ae4-b524-197073a5b229-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.986804 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.986784 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ad7dd378-440e-4ae4-b524-197073a5b229-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.991044 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.991020 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gll7z\" (UniqueName: \"kubernetes.io/projected/ad7dd378-440e-4ae4-b524-197073a5b229-kube-api-access-gll7z\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:37.991139 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:37.991118 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ad7dd378-440e-4ae4-b524-197073a5b229-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-hxshd\" (UID: \"ad7dd378-440e-4ae4-b524-197073a5b229\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:38.183436 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:38.183351 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:38.320998 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:38.320969 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd"] Jul 14 00:55:38.323287 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:55:38.323259 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad7dd378_440e_4ae4_b524_197073a5b229.slice/crio-bab9c2c1b0428e4f7ca883455504680b0e7bfe45746cc6f05b63acf135a827e0 WatchSource:0}: Error finding container bab9c2c1b0428e4f7ca883455504680b0e7bfe45746cc6f05b63acf135a827e0: Status 404 returned error can't find the container with id bab9c2c1b0428e4f7ca883455504680b0e7bfe45746cc6f05b63acf135a827e0 Jul 14 00:55:38.325725 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:38.325696 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:55:38.325822 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:38.325770 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:55:38.325886 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:38.325817 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:55:38.376594 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:38.376567 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" event={"ID":"ad7dd378-440e-4ae4-b524-197073a5b229","Type":"ContainerStarted","Data":"bab9c2c1b0428e4f7ca883455504680b0e7bfe45746cc6f05b63acf135a827e0"} Jul 14 00:55:38.413821 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:38.413789 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:39.383567 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:39.383531 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" event={"ID":"ad7dd378-440e-4ae4-b524-197073a5b229","Type":"ContainerStarted","Data":"9d4efdd728dd9432a989500e4ac8e0ed17fdc05b4d20426455818e8be83fd024"} Jul 14 00:55:39.402816 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:39.402772 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" podStartSLOduration=2.40275698 podStartE2EDuration="2.40275698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:55:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:55:39.401610205 +0000 UTC m=+676.252977577" watchObservedRunningTime="2026-07-14 00:55:39.40275698 +0000 UTC m=+676.254124410" Jul 14 00:55:40.183850 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:40.183811 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:40.188413 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:40.188390 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:40.388402 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:40.388368 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:40.389318 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:40.389302 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:55:41.473818 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:41.473760 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:50.864833 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:50.864789 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:50.966475 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:50.966428 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:51.017025 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.016999 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:51.670528 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.670499 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-vtmmc"] Jul 14 00:55:51.673980 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.673962 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vtmmc" Jul 14 00:55:51.676090 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.676067 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-cscqq\"" Jul 14 00:55:51.681523 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.681499 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vtmmc"] Jul 14 00:55:51.703031 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.703006 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:51.790505 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.790469 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njq96\" (UniqueName: \"kubernetes.io/projected/8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7-kube-api-access-njq96\") pod \"authorino-58cd955675-vtmmc\" (UID: \"8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7\") " pod="kuadrant-system/authorino-58cd955675-vtmmc" Jul 14 00:55:51.791690 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.791668 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:55:51.792929 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.792911 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-gplsf"] Jul 14 00:55:51.796136 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.796118 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gplsf" Jul 14 00:55:51.803022 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.803001 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gplsf"] Jul 14 00:55:51.891146 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.891120 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njq96\" (UniqueName: \"kubernetes.io/projected/8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7-kube-api-access-njq96\") pod \"authorino-58cd955675-vtmmc\" (UID: \"8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7\") " pod="kuadrant-system/authorino-58cd955675-vtmmc" Jul 14 00:55:51.891470 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.891219 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dql6h\" (UniqueName: \"kubernetes.io/projected/e34cc094-0869-43ce-b038-18738d62c814-kube-api-access-dql6h\") pod \"authorino-75d556bd77-gplsf\" (UID: \"e34cc094-0869-43ce-b038-18738d62c814\") " pod="kuadrant-system/authorino-75d556bd77-gplsf" Jul 14 00:55:51.897866 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.897846 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njq96\" (UniqueName: \"kubernetes.io/projected/8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7-kube-api-access-njq96\") pod \"authorino-58cd955675-vtmmc\" (UID: \"8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7\") " pod="kuadrant-system/authorino-58cd955675-vtmmc" Jul 14 00:55:51.986070 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.986022 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vtmmc" Jul 14 00:55:51.992632 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.992613 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dql6h\" (UniqueName: \"kubernetes.io/projected/e34cc094-0869-43ce-b038-18738d62c814-kube-api-access-dql6h\") pod \"authorino-75d556bd77-gplsf\" (UID: \"e34cc094-0869-43ce-b038-18738d62c814\") " pod="kuadrant-system/authorino-75d556bd77-gplsf" Jul 14 00:55:51.999661 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:51.999640 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dql6h\" (UniqueName: \"kubernetes.io/projected/e34cc094-0869-43ce-b038-18738d62c814-kube-api-access-dql6h\") pod \"authorino-75d556bd77-gplsf\" (UID: \"e34cc094-0869-43ce-b038-18738d62c814\") " pod="kuadrant-system/authorino-75d556bd77-gplsf" Jul 14 00:55:52.105349 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:52.105317 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gplsf" Jul 14 00:55:52.118357 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:52.118331 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vtmmc"] Jul 14 00:55:52.119321 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:55:52.119262 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8efa3ab5_edcc_4dbf_9104_ad35e18b0bb7.slice/crio-15083cdd71afaee9ab2dec7af807e671b6cb7d9b0d5ab67506580548644b1813 WatchSource:0}: Error finding container 15083cdd71afaee9ab2dec7af807e671b6cb7d9b0d5ab67506580548644b1813: Status 404 returned error can't find the container with id 15083cdd71afaee9ab2dec7af807e671b6cb7d9b0d5ab67506580548644b1813 Jul 14 00:55:52.245273 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:52.245207 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gplsf"] Jul 14 00:55:52.248066 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:55:52.248040 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode34cc094_0869_43ce_b038_18738d62c814.slice/crio-21c68c4da46f167217c55ba256b130d219c4664de67d0c26f5c5cd48dcfe66d7 WatchSource:0}: Error finding container 21c68c4da46f167217c55ba256b130d219c4664de67d0c26f5c5cd48dcfe66d7: Status 404 returned error can't find the container with id 21c68c4da46f167217c55ba256b130d219c4664de67d0c26f5c5cd48dcfe66d7 Jul 14 00:55:52.452058 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:52.452021 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vtmmc" event={"ID":"8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7","Type":"ContainerStarted","Data":"15083cdd71afaee9ab2dec7af807e671b6cb7d9b0d5ab67506580548644b1813"} Jul 14 00:55:52.453270 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:52.453239 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gplsf" event={"ID":"e34cc094-0869-43ce-b038-18738d62c814","Type":"ContainerStarted","Data":"21c68c4da46f167217c55ba256b130d219c4664de67d0c26f5c5cd48dcfe66d7"} Jul 14 00:55:56.482672 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:56.482577 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vtmmc" event={"ID":"8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7","Type":"ContainerStarted","Data":"17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102"} Jul 14 00:55:56.484215 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:56.484189 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gplsf" event={"ID":"e34cc094-0869-43ce-b038-18738d62c814","Type":"ContainerStarted","Data":"b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164"} Jul 14 00:55:56.497068 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:56.497022 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-vtmmc" podStartSLOduration=1.560038796 podStartE2EDuration="5.497008194s" totalImagesPullingTime="3.936969398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:55:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:55:56.496284763 +0000 UTC m=+693.347652134" watchObservedRunningTime="2026-07-14 00:55:56.497008194 +0000 UTC m=+693.348375601" Jul 14 00:55:56.509235 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:56.509185 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-gplsf" podStartSLOduration=1.7122789470000002 podStartE2EDuration="5.509168813s" totalImagesPullingTime="3.796889866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:55:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:55:56.508686463 +0000 UTC m=+693.360053834" watchObservedRunningTime="2026-07-14 00:55:56.509168813 +0000 UTC m=+693.360536184" Jul 14 00:55:56.537750 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:56.537682 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vtmmc"] Jul 14 00:55:58.501113 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:58.501060 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-vtmmc" podUID="8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7" containerName="authorino" containerID="cri-o://17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102" gracePeriod=30 Jul 14 00:55:58.747048 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:58.747023 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vtmmc" Jul 14 00:55:58.867378 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:58.867356 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njq96\" (UniqueName: \"kubernetes.io/projected/8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7-kube-api-access-njq96\") pod \"8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7\" (UID: \"8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7\") " Jul 14 00:55:58.869945 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:58.869918 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7-kube-api-access-njq96" (OuterVolumeSpecName: "kube-api-access-njq96") pod "8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7" (UID: "8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7"). InnerVolumeSpecName "kube-api-access-njq96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:55:58.968829 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:58.968793 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njq96\" (UniqueName: \"kubernetes.io/projected/8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7-kube-api-access-njq96\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:55:59.507905 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:59.507868 2557 generic.go:356] "Generic (PLEG): container finished" podID="8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7" containerID="17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102" exitCode=0 Jul 14 00:55:59.508267 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:59.507917 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vtmmc" Jul 14 00:55:59.508267 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:59.507961 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vtmmc" event={"ID":"8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7","Type":"ContainerDied","Data":"17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102"} Jul 14 00:55:59.508267 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:59.508017 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vtmmc" event={"ID":"8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7","Type":"ContainerDied","Data":"15083cdd71afaee9ab2dec7af807e671b6cb7d9b0d5ab67506580548644b1813"} Jul 14 00:55:59.508267 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:59.508039 2557 scope.go:117] "RemoveContainer" containerID="17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102" Jul 14 00:55:59.517402 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:59.517384 2557 scope.go:117] "RemoveContainer" containerID="17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102" Jul 14 00:55:59.517689 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:55:59.517669 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102\": container with ID starting with 17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102 not found: ID does not exist" containerID="17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102" Jul 14 00:55:59.517733 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:59.517697 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102"} err="failed to get container status \"17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102\": rpc error: code = NotFound desc = could not find container \"17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102\": container with ID starting with 17ad8604703d86d1267ef5183f5506914a5ac26e96aab87de4bfa813617e8102 not found: ID does not exist" Jul 14 00:55:59.527425 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:59.527399 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vtmmc"] Jul 14 00:55:59.530879 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:59.530857 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-vtmmc"] Jul 14 00:55:59.677562 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:55:59.677540 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7" path="/var/lib/kubelet/pods/8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7/volumes" Jul 14 00:56:04.781045 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:04.781003 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:04.881233 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:04.881195 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:11.940363 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:11.940308 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:12.043691 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:12.043662 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:12.449050 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:12.448972 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:25.178805 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.178758 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:25.414568 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.414534 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-r6wrb"] Jul 14 00:56:25.415164 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.415125 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7" containerName="authorino" Jul 14 00:56:25.415164 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.415162 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7" containerName="authorino" Jul 14 00:56:25.415326 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.415242 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8efa3ab5-edcc-4dbf-9104-ad35e18b0bb7" containerName="authorino" Jul 14 00:56:25.419400 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.419380 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-r6wrb" Jul 14 00:56:25.424512 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.424491 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-r6wrb"] Jul 14 00:56:25.471973 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.471920 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fb4f\" (UniqueName: \"kubernetes.io/projected/55ff1173-07e2-4d67-ab6d-11f68d0caa6c-kube-api-access-4fb4f\") pod \"authorino-5d4b67c446-r6wrb\" (UID: \"55ff1173-07e2-4d67-ab6d-11f68d0caa6c\") " pod="kuadrant-system/authorino-5d4b67c446-r6wrb" Jul 14 00:56:25.543599 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.543574 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:25.572441 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.572417 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fb4f\" (UniqueName: \"kubernetes.io/projected/55ff1173-07e2-4d67-ab6d-11f68d0caa6c-kube-api-access-4fb4f\") pod \"authorino-5d4b67c446-r6wrb\" (UID: \"55ff1173-07e2-4d67-ab6d-11f68d0caa6c\") " pod="kuadrant-system/authorino-5d4b67c446-r6wrb" Jul 14 00:56:25.579757 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.579733 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fb4f\" (UniqueName: \"kubernetes.io/projected/55ff1173-07e2-4d67-ab6d-11f68d0caa6c-kube-api-access-4fb4f\") pod \"authorino-5d4b67c446-r6wrb\" (UID: \"55ff1173-07e2-4d67-ab6d-11f68d0caa6c\") " pod="kuadrant-system/authorino-5d4b67c446-r6wrb" Jul 14 00:56:25.660324 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.660303 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-r6wrb"] Jul 14 00:56:25.660519 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.660507 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-r6wrb" Jul 14 00:56:25.682310 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.682285 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55b988d5c9-cmnbg"] Jul 14 00:56:25.686007 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.685990 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b988d5c9-cmnbg" Jul 14 00:56:25.691448 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.691429 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55b988d5c9-cmnbg"] Jul 14 00:56:25.773911 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.773885 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rhgr\" (UniqueName: \"kubernetes.io/projected/6a91866d-e5ae-4834-9fd7-b3b55e7947bb-kube-api-access-4rhgr\") pod \"authorino-55b988d5c9-cmnbg\" (UID: \"6a91866d-e5ae-4834-9fd7-b3b55e7947bb\") " pod="kuadrant-system/authorino-55b988d5c9-cmnbg" Jul 14 00:56:25.784328 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.784296 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55b988d5c9-cmnbg"] Jul 14 00:56:25.784529 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:56:25.784509 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4rhgr], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-55b988d5c9-cmnbg" podUID="6a91866d-e5ae-4834-9fd7-b3b55e7947bb" Jul 14 00:56:25.795274 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.795252 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-r6wrb"] Jul 14 00:56:25.797801 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:56:25.797770 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55ff1173_07e2_4d67_ab6d_11f68d0caa6c.slice/crio-00fda6cadc2714409d0b34e1ee5cefa95e73b209abf3522cbf5422218985ba42 WatchSource:0}: Error finding container 00fda6cadc2714409d0b34e1ee5cefa95e73b209abf3522cbf5422218985ba42: Status 404 returned error can't find the container with id 00fda6cadc2714409d0b34e1ee5cefa95e73b209abf3522cbf5422218985ba42 Jul 14 00:56:25.802198 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.802179 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 00:56:25.807382 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.807360 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c6d4fdd8c-st2tp"] Jul 14 00:56:25.810655 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.810639 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" Jul 14 00:56:25.812469 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.812451 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 14 00:56:25.817057 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.817040 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c6d4fdd8c-st2tp"] Jul 14 00:56:25.875190 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.875143 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rhgr\" (UniqueName: \"kubernetes.io/projected/6a91866d-e5ae-4834-9fd7-b3b55e7947bb-kube-api-access-4rhgr\") pod \"authorino-55b988d5c9-cmnbg\" (UID: \"6a91866d-e5ae-4834-9fd7-b3b55e7947bb\") " pod="kuadrant-system/authorino-55b988d5c9-cmnbg" Jul 14 00:56:25.875355 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.875216 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c59e793c-c62e-4577-a87d-7f8adc429add-tls-cert\") pod \"authorino-6c6d4fdd8c-st2tp\" (UID: \"c59e793c-c62e-4577-a87d-7f8adc429add\") " pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" Jul 14 00:56:25.875355 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.875243 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp722\" (UniqueName: \"kubernetes.io/projected/c59e793c-c62e-4577-a87d-7f8adc429add-kube-api-access-sp722\") pod \"authorino-6c6d4fdd8c-st2tp\" (UID: \"c59e793c-c62e-4577-a87d-7f8adc429add\") " pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" Jul 14 00:56:25.882566 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.882537 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rhgr\" (UniqueName: \"kubernetes.io/projected/6a91866d-e5ae-4834-9fd7-b3b55e7947bb-kube-api-access-4rhgr\") pod \"authorino-55b988d5c9-cmnbg\" (UID: \"6a91866d-e5ae-4834-9fd7-b3b55e7947bb\") " pod="kuadrant-system/authorino-55b988d5c9-cmnbg" Jul 14 00:56:25.976030 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.976009 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c59e793c-c62e-4577-a87d-7f8adc429add-tls-cert\") pod \"authorino-6c6d4fdd8c-st2tp\" (UID: \"c59e793c-c62e-4577-a87d-7f8adc429add\") " pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" Jul 14 00:56:25.976138 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.976043 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp722\" (UniqueName: \"kubernetes.io/projected/c59e793c-c62e-4577-a87d-7f8adc429add-kube-api-access-sp722\") pod \"authorino-6c6d4fdd8c-st2tp\" (UID: \"c59e793c-c62e-4577-a87d-7f8adc429add\") " pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" Jul 14 00:56:25.978173 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.978137 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c59e793c-c62e-4577-a87d-7f8adc429add-tls-cert\") pod \"authorino-6c6d4fdd8c-st2tp\" (UID: \"c59e793c-c62e-4577-a87d-7f8adc429add\") " pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" Jul 14 00:56:25.982868 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:25.982846 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp722\" (UniqueName: \"kubernetes.io/projected/c59e793c-c62e-4577-a87d-7f8adc429add-kube-api-access-sp722\") pod \"authorino-6c6d4fdd8c-st2tp\" (UID: \"c59e793c-c62e-4577-a87d-7f8adc429add\") " pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" Jul 14 00:56:26.121044 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.121023 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" Jul 14 00:56:26.257389 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.257360 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c6d4fdd8c-st2tp"] Jul 14 00:56:26.258229 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:56:26.258205 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc59e793c_c62e_4577_a87d_7f8adc429add.slice/crio-830f52ed8d9cc635ab2bb9761dfde1820ca1b7aa4befb2d469b83b430fd81421 WatchSource:0}: Error finding container 830f52ed8d9cc635ab2bb9761dfde1820ca1b7aa4befb2d469b83b430fd81421: Status 404 returned error can't find the container with id 830f52ed8d9cc635ab2bb9761dfde1820ca1b7aa4befb2d469b83b430fd81421 Jul 14 00:56:26.647942 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.647912 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" event={"ID":"c59e793c-c62e-4577-a87d-7f8adc429add","Type":"ContainerStarted","Data":"830f52ed8d9cc635ab2bb9761dfde1820ca1b7aa4befb2d469b83b430fd81421"} Jul 14 00:56:26.649315 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.649293 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b988d5c9-cmnbg" Jul 14 00:56:26.649459 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.649324 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-r6wrb" event={"ID":"55ff1173-07e2-4d67-ab6d-11f68d0caa6c","Type":"ContainerStarted","Data":"48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7"} Jul 14 00:56:26.649459 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.649344 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-r6wrb" podUID="55ff1173-07e2-4d67-ab6d-11f68d0caa6c" containerName="authorino" containerID="cri-o://48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7" gracePeriod=30 Jul 14 00:56:26.649459 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.649354 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-r6wrb" event={"ID":"55ff1173-07e2-4d67-ab6d-11f68d0caa6c","Type":"ContainerStarted","Data":"00fda6cadc2714409d0b34e1ee5cefa95e73b209abf3522cbf5422218985ba42"} Jul 14 00:56:26.656868 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.656845 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b988d5c9-cmnbg" Jul 14 00:56:26.665129 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.665088 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-r6wrb" podStartSLOduration=1.2778852569999999 podStartE2EDuration="1.665075297s" totalImagesPullingTime="387.19004ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:56:26.662882697 +0000 UTC m=+723.514250068" watchObservedRunningTime="2026-07-14 00:56:26.665075297 +0000 UTC m=+723.516442672" Jul 14 00:56:26.681449 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.681429 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rhgr\" (UniqueName: \"kubernetes.io/projected/6a91866d-e5ae-4834-9fd7-b3b55e7947bb-kube-api-access-4rhgr\") pod \"6a91866d-e5ae-4834-9fd7-b3b55e7947bb\" (UID: \"6a91866d-e5ae-4834-9fd7-b3b55e7947bb\") " Jul 14 00:56:26.683357 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.683338 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a91866d-e5ae-4834-9fd7-b3b55e7947bb-kube-api-access-4rhgr" (OuterVolumeSpecName: "kube-api-access-4rhgr") pod "6a91866d-e5ae-4834-9fd7-b3b55e7947bb" (UID: "6a91866d-e5ae-4834-9fd7-b3b55e7947bb"). InnerVolumeSpecName "kube-api-access-4rhgr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:56:26.782440 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.782417 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rhgr\" (UniqueName: \"kubernetes.io/projected/6a91866d-e5ae-4834-9fd7-b3b55e7947bb-kube-api-access-4rhgr\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:56:26.913961 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.913939 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-r6wrb" Jul 14 00:56:26.984209 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.984184 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fb4f\" (UniqueName: \"kubernetes.io/projected/55ff1173-07e2-4d67-ab6d-11f68d0caa6c-kube-api-access-4fb4f\") pod \"55ff1173-07e2-4d67-ab6d-11f68d0caa6c\" (UID: \"55ff1173-07e2-4d67-ab6d-11f68d0caa6c\") " Jul 14 00:56:26.986072 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:26.986051 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ff1173-07e2-4d67-ab6d-11f68d0caa6c-kube-api-access-4fb4f" (OuterVolumeSpecName: "kube-api-access-4fb4f") pod "55ff1173-07e2-4d67-ab6d-11f68d0caa6c" (UID: "55ff1173-07e2-4d67-ab6d-11f68d0caa6c"). InnerVolumeSpecName "kube-api-access-4fb4f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:56:27.085174 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.085137 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fb4f\" (UniqueName: \"kubernetes.io/projected/55ff1173-07e2-4d67-ab6d-11f68d0caa6c-kube-api-access-4fb4f\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:56:27.656088 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.656058 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" event={"ID":"c59e793c-c62e-4577-a87d-7f8adc429add","Type":"ContainerStarted","Data":"886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86"} Jul 14 00:56:27.657288 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.657259 2557 generic.go:356] "Generic (PLEG): container finished" podID="55ff1173-07e2-4d67-ab6d-11f68d0caa6c" containerID="48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7" exitCode=0 Jul 14 00:56:27.657383 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.657291 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-r6wrb" Jul 14 00:56:27.657383 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.657338 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-r6wrb" event={"ID":"55ff1173-07e2-4d67-ab6d-11f68d0caa6c","Type":"ContainerDied","Data":"48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7"} Jul 14 00:56:27.657383 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.657372 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-r6wrb" event={"ID":"55ff1173-07e2-4d67-ab6d-11f68d0caa6c","Type":"ContainerDied","Data":"00fda6cadc2714409d0b34e1ee5cefa95e73b209abf3522cbf5422218985ba42"} Jul 14 00:56:27.657493 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.657391 2557 scope.go:117] "RemoveContainer" containerID="48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7" Jul 14 00:56:27.657526 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.657492 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55b988d5c9-cmnbg" Jul 14 00:56:27.667226 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.667120 2557 scope.go:117] "RemoveContainer" containerID="48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7" Jul 14 00:56:27.667535 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:56:27.667508 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7\": container with ID starting with 48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7 not found: ID does not exist" containerID="48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7" Jul 14 00:56:27.667616 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.667546 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7"} err="failed to get container status \"48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7\": rpc error: code = NotFound desc = could not find container \"48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7\": container with ID starting with 48ddb8df9f0e8483c211f2955ca8dee5eb33fda659db781a87ec1874a733b6d7 not found: ID does not exist" Jul 14 00:56:27.670846 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.670806 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" podStartSLOduration=2.246026999 podStartE2EDuration="2.670793844s" totalImagesPullingTime="424.766845ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:56:27.669108801 +0000 UTC m=+724.520476170" watchObservedRunningTime="2026-07-14 00:56:27.670793844 +0000 UTC m=+724.522161256" Jul 14 00:56:27.693693 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.693670 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55b988d5c9-cmnbg"] Jul 14 00:56:27.699288 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.699245 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gplsf"] Jul 14 00:56:27.699437 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.699409 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-gplsf" podUID="e34cc094-0869-43ce-b038-18738d62c814" containerName="authorino" containerID="cri-o://b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164" gracePeriod=30 Jul 14 00:56:27.701792 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.701773 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-55b988d5c9-cmnbg"] Jul 14 00:56:27.714244 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.714223 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-r6wrb"] Jul 14 00:56:27.720373 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.720354 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-r6wrb"] Jul 14 00:56:27.955066 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.955040 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gplsf" Jul 14 00:56:27.993114 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.993093 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dql6h\" (UniqueName: \"kubernetes.io/projected/e34cc094-0869-43ce-b038-18738d62c814-kube-api-access-dql6h\") pod \"e34cc094-0869-43ce-b038-18738d62c814\" (UID: \"e34cc094-0869-43ce-b038-18738d62c814\") " Jul 14 00:56:27.994985 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:27.994965 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e34cc094-0869-43ce-b038-18738d62c814-kube-api-access-dql6h" (OuterVolumeSpecName: "kube-api-access-dql6h") pod "e34cc094-0869-43ce-b038-18738d62c814" (UID: "e34cc094-0869-43ce-b038-18738d62c814"). InnerVolumeSpecName "kube-api-access-dql6h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:56:28.094633 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:28.094608 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dql6h\" (UniqueName: \"kubernetes.io/projected/e34cc094-0869-43ce-b038-18738d62c814-kube-api-access-dql6h\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:56:28.664220 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:28.664190 2557 generic.go:356] "Generic (PLEG): container finished" podID="e34cc094-0869-43ce-b038-18738d62c814" containerID="b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164" exitCode=0 Jul 14 00:56:28.664630 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:28.664229 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-gplsf" Jul 14 00:56:28.664630 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:28.664269 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gplsf" event={"ID":"e34cc094-0869-43ce-b038-18738d62c814","Type":"ContainerDied","Data":"b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164"} Jul 14 00:56:28.664630 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:28.664311 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-gplsf" event={"ID":"e34cc094-0869-43ce-b038-18738d62c814","Type":"ContainerDied","Data":"21c68c4da46f167217c55ba256b130d219c4664de67d0c26f5c5cd48dcfe66d7"} Jul 14 00:56:28.664630 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:28.664329 2557 scope.go:117] "RemoveContainer" containerID="b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164" Jul 14 00:56:28.673656 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:28.673641 2557 scope.go:117] "RemoveContainer" containerID="b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164" Jul 14 00:56:28.673933 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:56:28.673914 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164\": container with ID starting with b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164 not found: ID does not exist" containerID="b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164" Jul 14 00:56:28.673985 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:28.673941 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164"} err="failed to get container status \"b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164\": rpc error: code = NotFound desc = could not find container \"b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164\": container with ID starting with b53b5e2cce6452c36e501f7d5d3675d8f8dece0300f8355ab7640fcce1672164 not found: ID does not exist" Jul 14 00:56:28.683945 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:28.683923 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gplsf"] Jul 14 00:56:28.685938 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:28.685920 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-gplsf"] Jul 14 00:56:29.677034 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:29.677003 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55ff1173-07e2-4d67-ab6d-11f68d0caa6c" path="/var/lib/kubelet/pods/55ff1173-07e2-4d67-ab6d-11f68d0caa6c/volumes" Jul 14 00:56:29.677397 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:29.677329 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a91866d-e5ae-4834-9fd7-b3b55e7947bb" path="/var/lib/kubelet/pods/6a91866d-e5ae-4834-9fd7-b3b55e7947bb/volumes" Jul 14 00:56:29.677531 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:29.677518 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e34cc094-0869-43ce-b038-18738d62c814" path="/var/lib/kubelet/pods/e34cc094-0869-43ce-b038-18738d62c814/volumes" Jul 14 00:56:32.398820 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:32.398788 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:32.498703 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:32.498663 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:32.557316 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:32.557288 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:35.532119 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:35.532041 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:35.630474 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:35.630441 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:41.542007 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:41.541974 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:56:41.589873 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:56:41.589835 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:07.263304 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:07.263273 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:07.364728 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:07.364690 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:07.835466 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:07.835439 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:07.956311 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:07.956284 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:08.413162 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.413117 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-jlqgh"] Jul 14 00:57:08.413580 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.413506 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e34cc094-0869-43ce-b038-18738d62c814" containerName="authorino" Jul 14 00:57:08.413580 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.413520 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e34cc094-0869-43ce-b038-18738d62c814" containerName="authorino" Jul 14 00:57:08.413580 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.413549 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ff1173-07e2-4d67-ab6d-11f68d0caa6c" containerName="authorino" Jul 14 00:57:08.413580 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.413557 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ff1173-07e2-4d67-ab6d-11f68d0caa6c" containerName="authorino" Jul 14 00:57:08.413776 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.413618 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e34cc094-0869-43ce-b038-18738d62c814" containerName="authorino" Jul 14 00:57:08.413776 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.413637 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ff1173-07e2-4d67-ab6d-11f68d0caa6c" containerName="authorino" Jul 14 00:57:08.416139 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.416116 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:08.417311 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.417294 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" Jul 14 00:57:08.419285 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.419266 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 14 00:57:08.419395 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.419267 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-ksgjh\"" Jul 14 00:57:08.424580 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.424245 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-jlqgh"] Jul 14 00:57:08.492461 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.492437 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jkqp\" (UniqueName: \"kubernetes.io/projected/faf77f07-737b-4772-961a-4fdfd8432cea-kube-api-access-9jkqp\") pod \"payload-processing-f9b8d9b9b-jlqgh\" (UID: \"faf77f07-737b-4772-961a-4fdfd8432cea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" Jul 14 00:57:08.492579 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.492544 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf77f07-737b-4772-961a-4fdfd8432cea-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-jlqgh\" (UID: \"faf77f07-737b-4772-961a-4fdfd8432cea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" Jul 14 00:57:08.514445 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.514413 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:08.593912 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.593886 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jkqp\" (UniqueName: \"kubernetes.io/projected/faf77f07-737b-4772-961a-4fdfd8432cea-kube-api-access-9jkqp\") pod \"payload-processing-f9b8d9b9b-jlqgh\" (UID: \"faf77f07-737b-4772-961a-4fdfd8432cea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" Jul 14 00:57:08.594081 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.594062 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf77f07-737b-4772-961a-4fdfd8432cea-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-jlqgh\" (UID: \"faf77f07-737b-4772-961a-4fdfd8432cea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" Jul 14 00:57:08.594833 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.594812 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf77f07-737b-4772-961a-4fdfd8432cea-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-jlqgh\" (UID: \"faf77f07-737b-4772-961a-4fdfd8432cea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" Jul 14 00:57:08.600722 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.600698 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jkqp\" (UniqueName: \"kubernetes.io/projected/faf77f07-737b-4772-961a-4fdfd8432cea-kube-api-access-9jkqp\") pod \"payload-processing-f9b8d9b9b-jlqgh\" (UID: \"faf77f07-737b-4772-961a-4fdfd8432cea\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" Jul 14 00:57:08.729503 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.729444 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" Jul 14 00:57:08.866352 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:08.866323 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-jlqgh"] Jul 14 00:57:08.868216 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:57:08.868179 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf77f07_737b_4772_961a_4fdfd8432cea.slice/crio-b56453c4856daffd95416d577d8cc5fba2a3001eaa1addec94e0d8728a05ebce WatchSource:0}: Error finding container b56453c4856daffd95416d577d8cc5fba2a3001eaa1addec94e0d8728a05ebce: Status 404 returned error can't find the container with id b56453c4856daffd95416d577d8cc5fba2a3001eaa1addec94e0d8728a05ebce Jul 14 00:57:09.901284 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:09.901243 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" event={"ID":"faf77f07-737b-4772-961a-4fdfd8432cea","Type":"ContainerStarted","Data":"b56453c4856daffd95416d577d8cc5fba2a3001eaa1addec94e0d8728a05ebce"} Jul 14 00:57:10.908191 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:10.908135 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" event={"ID":"faf77f07-737b-4772-961a-4fdfd8432cea","Type":"ContainerStarted","Data":"ab209d74fe7d5d8adefeb91824940caf62d66bae3cc9ec0b5caee76a0b3eb205"} Jul 14 00:57:10.908591 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:10.908323 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" Jul 14 00:57:10.921632 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:10.921577 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" podStartSLOduration=1.354615934 podStartE2EDuration="2.921564361s" totalImagesPullingTime="1.566948427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:57:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:57:10.921186603 +0000 UTC m=+767.772553973" watchObservedRunningTime="2026-07-14 00:57:10.921564361 +0000 UTC m=+767.772931731" Jul 14 00:57:15.937528 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:15.937474 2557 generic.go:356] "Generic (PLEG): container finished" podID="ad7dd378-440e-4ae4-b524-197073a5b229" containerID="9d4efdd728dd9432a989500e4ac8e0ed17fdc05b4d20426455818e8be83fd024" exitCode=0 Jul 14 00:57:15.937846 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:15.937547 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" event={"ID":"ad7dd378-440e-4ae4-b524-197073a5b229","Type":"ContainerDied","Data":"9d4efdd728dd9432a989500e4ac8e0ed17fdc05b4d20426455818e8be83fd024"} Jul 14 00:57:15.937914 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:15.937899 2557 scope.go:117] "RemoveContainer" containerID="9d4efdd728dd9432a989500e4ac8e0ed17fdc05b4d20426455818e8be83fd024" Jul 14 00:57:15.938814 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:15.938788 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:57:15.938874 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:15.938854 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:57:15.938913 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:15.938888 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 00:57:16.944357 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:16.944319 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" event={"ID":"ad7dd378-440e-4ae4-b524-197073a5b229","Type":"ContainerStarted","Data":"03adf2b05a61f3ed7b8529c4f1e7a1f8438c01c2d85ab5a2e2743a121751245f"} Jul 14 00:57:17.187171 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:17.186554 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:57:17.187171 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:17.186727 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hxshd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 14 00:57:17.187171 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:17.186774 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" podUID="ad7dd378-440e-4ae4-b524-197073a5b229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 14 00:57:18.184637 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:18.184602 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hxshd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 14 00:57:18.185216 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:18.185185 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" podUID="ad7dd378-440e-4ae4-b524-197073a5b229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 14 00:57:18.607122 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:18.607089 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:18.804084 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:18.804051 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:19.183925 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:19.183889 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hxshd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 14 00:57:19.184216 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:19.184184 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" podUID="ad7dd378-440e-4ae4-b524-197073a5b229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 14 00:57:20.184522 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:20.184487 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-hxshd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 14 00:57:20.185084 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:20.185053 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" podUID="ad7dd378-440e-4ae4-b524-197073a5b229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 14 00:57:21.187604 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:21.187577 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:57:21.187947 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:21.187818 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:57:21.188545 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:21.188530 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-hxshd" Jul 14 00:57:21.915145 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:21.915113 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-jlqgh" Jul 14 00:57:25.660287 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:25.660253 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:36.529689 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.529660 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz"] Jul 14 00:57:36.534942 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.534919 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.536767 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.536748 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 00:57:36.536868 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.536780 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k75jk\"" Jul 14 00:57:36.537313 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.537297 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 00:57:36.537403 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.537389 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 14 00:57:36.540966 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.540943 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.541075 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.540972 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxvgb\" (UniqueName: \"kubernetes.io/projected/3978cfae-8977-4685-9e59-c1c3d1659fda-kube-api-access-qxvgb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.541075 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.540994 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.541075 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.541034 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.541273 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.541137 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.541273 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.541195 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3978cfae-8977-4685-9e59-c1c3d1659fda-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.545941 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.545922 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz"] Jul 14 00:57:36.557483 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.557462 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:36.642204 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.642178 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.642350 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.642231 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3978cfae-8977-4685-9e59-c1c3d1659fda-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.642350 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.642283 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.642519 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.642496 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxvgb\" (UniqueName: \"kubernetes.io/projected/3978cfae-8977-4685-9e59-c1c3d1659fda-kube-api-access-qxvgb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.642603 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.642547 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.642603 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.642579 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.642770 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.642751 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.642918 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.642896 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.643095 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.643074 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.644685 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.644665 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3978cfae-8977-4685-9e59-c1c3d1659fda-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.644940 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.644924 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3978cfae-8977-4685-9e59-c1c3d1659fda-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.650042 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.650021 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxvgb\" (UniqueName: \"kubernetes.io/projected/3978cfae-8977-4685-9e59-c1c3d1659fda-kube-api-access-qxvgb\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vgnzz\" (UID: \"3978cfae-8977-4685-9e59-c1c3d1659fda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.845825 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.845792 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:36.992299 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:36.992272 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz"] Jul 14 00:57:36.993811 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:57:36.993789 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3978cfae_8977_4685_9e59_c1c3d1659fda.slice/crio-1381c3715792817ae8117d6870eeb816fd977df4e6b3a242ed93321272ee8793 WatchSource:0}: Error finding container 1381c3715792817ae8117d6870eeb816fd977df4e6b3a242ed93321272ee8793: Status 404 returned error can't find the container with id 1381c3715792817ae8117d6870eeb816fd977df4e6b3a242ed93321272ee8793 Jul 14 00:57:37.046794 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:37.046764 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" event={"ID":"3978cfae-8977-4685-9e59-c1c3d1659fda","Type":"ContainerStarted","Data":"1381c3715792817ae8117d6870eeb816fd977df4e6b3a242ed93321272ee8793"} Jul 14 00:57:39.061422 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:39.061377 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" event={"ID":"3978cfae-8977-4685-9e59-c1c3d1659fda","Type":"ContainerStarted","Data":"ecf3f4bd1e852b148ab7934341926663045fe77b9deb15fd822eeecb07ed90c5"} Jul 14 00:57:39.080371 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:39.080316 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" podStartSLOduration=1.965303611 podStartE2EDuration="3.080298719s" totalImagesPullingTime="1.114995108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:57:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:57:39.078847442 +0000 UTC m=+795.930214811" watchObservedRunningTime="2026-07-14 00:57:39.080298719 +0000 UTC m=+795.931666089" Jul 14 00:57:46.846115 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:46.846030 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:46.846115 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:46.846093 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:46.858438 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:46.858403 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:47.114763 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:47.114700 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vgnzz" Jul 14 00:57:57.041881 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.041846 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4"] Jul 14 00:57:57.048745 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.048726 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.050443 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.050425 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 00:57:57.055911 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.055693 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4"] Jul 14 00:57:57.071599 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.071555 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:57:57.107251 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.107224 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.107337 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.107266 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.107337 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.107308 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.107421 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.107336 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-588qx\" (UniqueName: \"kubernetes.io/projected/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-kube-api-access-588qx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.107421 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.107404 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.107507 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.107447 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.208809 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.208777 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.208970 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.208870 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.208970 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.208902 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-588qx\" (UniqueName: \"kubernetes.io/projected/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-kube-api-access-588qx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.208970 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.208935 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.209171 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.208969 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.209171 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.209012 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.209371 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.209353 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.209428 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.209410 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.209477 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.209428 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.211645 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.211622 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.211907 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.211891 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.215671 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.215650 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-588qx\" (UniqueName: \"kubernetes.io/projected/c95978e5-862d-4e87-ac1b-5cf1e949e6f4-kube-api-access-588qx\") pod \"e2e-trlp-test-simulated-kserve-598466548d-vzhj4\" (UID: \"c95978e5-862d-4e87-ac1b-5cf1e949e6f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.359723 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.359671 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:57:57.492051 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:57.492028 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4"] Jul 14 00:57:57.493019 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:57:57.492994 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95978e5_862d_4e87_ac1b_5cf1e949e6f4.slice/crio-f461c660ab03884c79462c288d6e0bed22b8acf16701dae248dbe372a51af799 WatchSource:0}: Error finding container f461c660ab03884c79462c288d6e0bed22b8acf16701dae248dbe372a51af799: Status 404 returned error can't find the container with id f461c660ab03884c79462c288d6e0bed22b8acf16701dae248dbe372a51af799 Jul 14 00:57:58.164582 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:58.164520 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" event={"ID":"c95978e5-862d-4e87-ac1b-5cf1e949e6f4","Type":"ContainerStarted","Data":"98eb73a5bf7d045adb5919202b0a432ccff76fcc256adbbc24b9255f9927eb72"} Jul 14 00:57:58.165002 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:58.164603 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" event={"ID":"c95978e5-862d-4e87-ac1b-5cf1e949e6f4","Type":"ContainerStarted","Data":"f461c660ab03884c79462c288d6e0bed22b8acf16701dae248dbe372a51af799"} Jul 14 00:57:58.180567 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:58.180520 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" podStartSLOduration=0.986252933 podStartE2EDuration="1.180504689s" totalImagesPullingTime="194.251756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:57:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:57:58.178881763 +0000 UTC m=+815.030249123" watchObservedRunningTime="2026-07-14 00:57:58.180504689 +0000 UTC m=+815.031872058" Jul 14 00:57:58.302551 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:57:58.302516 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:58:07.360275 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:07.360238 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:58:07.360760 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:07.360305 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:58:07.372909 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:07.372889 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:58:08.227040 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:08.227013 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-vzhj4" Jul 14 00:58:13.249888 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:13.249854 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:58:20.131871 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:20.131835 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:58:34.934891 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:34.934858 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8"] Jul 14 00:58:34.938826 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:34.938809 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:34.940851 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:34.940828 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 14 00:58:34.947717 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:34.947696 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8"] Jul 14 00:58:34.960652 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:34.960625 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:58:35.023202 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.023178 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.023308 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.023207 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.023376 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.023298 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.023376 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.023344 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.023484 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.023378 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtdr2\" (UniqueName: \"kubernetes.io/projected/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-kube-api-access-mtdr2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.023484 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.023421 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.123798 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.123768 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.123940 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.123802 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.123940 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.123837 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.123940 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.123869 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.123940 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.123889 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtdr2\" (UniqueName: \"kubernetes.io/projected/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-kube-api-access-mtdr2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.123940 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.123932 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.124301 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.124280 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.124376 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.124314 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.124426 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.124374 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.126089 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.126058 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.126386 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.126367 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.131410 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.131387 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtdr2\" (UniqueName: \"kubernetes.io/projected/a53bd28d-b04a-4b49-8a2e-f1ab9c206d38-kube-api-access-mtdr2\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8\" (UID: \"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.248693 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.248633 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:35.381489 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:35.381458 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8"] Jul 14 00:58:35.381882 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:58:35.381856 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda53bd28d_b04a_4b49_8a2e_f1ab9c206d38.slice/crio-3ad3f9f4579412d50c7dee36624a050eb5b05af0bdcb87cc2b1a383851e07d46 WatchSource:0}: Error finding container 3ad3f9f4579412d50c7dee36624a050eb5b05af0bdcb87cc2b1a383851e07d46: Status 404 returned error can't find the container with id 3ad3f9f4579412d50c7dee36624a050eb5b05af0bdcb87cc2b1a383851e07d46 Jul 14 00:58:36.365855 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:36.365822 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" event={"ID":"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38","Type":"ContainerStarted","Data":"08ad53f3f97d04426ddc5cb8cc4c70bfc538e20eea2863c1f88689fafba9a21e"} Jul 14 00:58:36.366312 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:36.365863 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" event={"ID":"a53bd28d-b04a-4b49-8a2e-f1ab9c206d38","Type":"ContainerStarted","Data":"3ad3f9f4579412d50c7dee36624a050eb5b05af0bdcb87cc2b1a383851e07d46"} Jul 14 00:58:36.380868 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:36.380823 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" podStartSLOduration=2.213094756 podStartE2EDuration="2.38080672s" totalImagesPullingTime="167.711964ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:58:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:58:36.379699204 +0000 UTC m=+853.231066571" watchObservedRunningTime="2026-07-14 00:58:36.38080672 +0000 UTC m=+853.232174114" Jul 14 00:58:42.796942 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:42.796910 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:58:45.249272 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:45.249237 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:45.249644 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:45.249285 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:45.262226 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:45.262204 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:45.427538 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:45.427515 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8" Jul 14 00:58:48.036972 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:48.036939 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:58:48.089779 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:58:48.089754 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:59:23.578085 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:23.578057 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 00:59:23.583513 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:23.583483 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 00:59:44.003879 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.003839 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-95cf4bc4b-mhrkc"] Jul 14 00:59:44.007626 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.007603 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-95cf4bc4b-mhrkc" Jul 14 00:59:44.014036 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.014012 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-95cf4bc4b-mhrkc"] Jul 14 00:59:44.157023 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.156997 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aa0c837b-56c9-43a9-a055-718c2d2c08a6-tls-cert\") pod \"authorino-95cf4bc4b-mhrkc\" (UID: \"aa0c837b-56c9-43a9-a055-718c2d2c08a6\") " pod="kuadrant-system/authorino-95cf4bc4b-mhrkc" Jul 14 00:59:44.157197 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.157047 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkh97\" (UniqueName: \"kubernetes.io/projected/aa0c837b-56c9-43a9-a055-718c2d2c08a6-kube-api-access-hkh97\") pod \"authorino-95cf4bc4b-mhrkc\" (UID: \"aa0c837b-56c9-43a9-a055-718c2d2c08a6\") " pod="kuadrant-system/authorino-95cf4bc4b-mhrkc" Jul 14 00:59:44.258176 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.258099 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aa0c837b-56c9-43a9-a055-718c2d2c08a6-tls-cert\") pod \"authorino-95cf4bc4b-mhrkc\" (UID: \"aa0c837b-56c9-43a9-a055-718c2d2c08a6\") " pod="kuadrant-system/authorino-95cf4bc4b-mhrkc" Jul 14 00:59:44.258289 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.258176 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkh97\" (UniqueName: \"kubernetes.io/projected/aa0c837b-56c9-43a9-a055-718c2d2c08a6-kube-api-access-hkh97\") pod \"authorino-95cf4bc4b-mhrkc\" (UID: \"aa0c837b-56c9-43a9-a055-718c2d2c08a6\") " pod="kuadrant-system/authorino-95cf4bc4b-mhrkc" Jul 14 00:59:44.260615 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.260593 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/aa0c837b-56c9-43a9-a055-718c2d2c08a6-tls-cert\") pod \"authorino-95cf4bc4b-mhrkc\" (UID: \"aa0c837b-56c9-43a9-a055-718c2d2c08a6\") " pod="kuadrant-system/authorino-95cf4bc4b-mhrkc" Jul 14 00:59:44.264945 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.264916 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkh97\" (UniqueName: \"kubernetes.io/projected/aa0c837b-56c9-43a9-a055-718c2d2c08a6-kube-api-access-hkh97\") pod \"authorino-95cf4bc4b-mhrkc\" (UID: \"aa0c837b-56c9-43a9-a055-718c2d2c08a6\") " pod="kuadrant-system/authorino-95cf4bc4b-mhrkc" Jul 14 00:59:44.317646 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.317622 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-95cf4bc4b-mhrkc" Jul 14 00:59:44.442541 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.442495 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:59:44.457753 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.457733 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-95cf4bc4b-mhrkc"] Jul 14 00:59:44.459483 ip-10-0-134-143 kubenswrapper[2557]: W0714 00:59:44.459457 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa0c837b_56c9_43a9_a055_718c2d2c08a6.slice/crio-05e7a3e30e3f2ab44ca687c575005fda158d6f4b2e7854f407475e2b055c8cde WatchSource:0}: Error finding container 05e7a3e30e3f2ab44ca687c575005fda158d6f4b2e7854f407475e2b055c8cde: Status 404 returned error can't find the container with id 05e7a3e30e3f2ab44ca687c575005fda158d6f4b2e7854f407475e2b055c8cde Jul 14 00:59:44.719855 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:44.719830 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-95cf4bc4b-mhrkc" event={"ID":"aa0c837b-56c9-43a9-a055-718c2d2c08a6","Type":"ContainerStarted","Data":"05e7a3e30e3f2ab44ca687c575005fda158d6f4b2e7854f407475e2b055c8cde"} Jul 14 00:59:45.732065 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:45.732021 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-95cf4bc4b-mhrkc" event={"ID":"aa0c837b-56c9-43a9-a055-718c2d2c08a6","Type":"ContainerStarted","Data":"b1facf506a9896d7f7df6b85531fbd62b06180d54f312a4a2d0c981349bdfaf4"} Jul 14 00:59:45.749776 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:45.749717 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-95cf4bc4b-mhrkc" podStartSLOduration=2.413262027 podStartE2EDuration="2.749680324s" totalImagesPullingTime="336.418297ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 00:59:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 00:59:45.745468824 +0000 UTC m=+922.596836194" watchObservedRunningTime="2026-07-14 00:59:45.749680324 +0000 UTC m=+922.601047701" Jul 14 00:59:45.786466 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:45.785141 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c6d4fdd8c-st2tp"] Jul 14 00:59:45.786466 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:45.785392 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" podUID="c59e793c-c62e-4577-a87d-7f8adc429add" containerName="authorino" containerID="cri-o://886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86" gracePeriod=30 Jul 14 00:59:46.053182 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.053131 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" Jul 14 00:59:46.179192 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.178127 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c59e793c-c62e-4577-a87d-7f8adc429add-tls-cert\") pod \"c59e793c-c62e-4577-a87d-7f8adc429add\" (UID: \"c59e793c-c62e-4577-a87d-7f8adc429add\") " Jul 14 00:59:46.179192 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.178204 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sp722\" (UniqueName: \"kubernetes.io/projected/c59e793c-c62e-4577-a87d-7f8adc429add-kube-api-access-sp722\") pod \"c59e793c-c62e-4577-a87d-7f8adc429add\" (UID: \"c59e793c-c62e-4577-a87d-7f8adc429add\") " Jul 14 00:59:46.181738 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.181691 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c59e793c-c62e-4577-a87d-7f8adc429add-kube-api-access-sp722" (OuterVolumeSpecName: "kube-api-access-sp722") pod "c59e793c-c62e-4577-a87d-7f8adc429add" (UID: "c59e793c-c62e-4577-a87d-7f8adc429add"). InnerVolumeSpecName "kube-api-access-sp722". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 00:59:46.192210 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.192123 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c59e793c-c62e-4577-a87d-7f8adc429add-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c59e793c-c62e-4577-a87d-7f8adc429add" (UID: "c59e793c-c62e-4577-a87d-7f8adc429add"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 00:59:46.279883 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.279828 2557 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c59e793c-c62e-4577-a87d-7f8adc429add-tls-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:59:46.280051 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.279888 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp722\" (UniqueName: \"kubernetes.io/projected/c59e793c-c62e-4577-a87d-7f8adc429add-kube-api-access-sp722\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 00:59:46.738229 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.738187 2557 generic.go:356] "Generic (PLEG): container finished" podID="c59e793c-c62e-4577-a87d-7f8adc429add" containerID="886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86" exitCode=0 Jul 14 00:59:46.738657 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.738248 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" Jul 14 00:59:46.738657 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.738268 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" event={"ID":"c59e793c-c62e-4577-a87d-7f8adc429add","Type":"ContainerDied","Data":"886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86"} Jul 14 00:59:46.738657 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.738318 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c6d4fdd8c-st2tp" event={"ID":"c59e793c-c62e-4577-a87d-7f8adc429add","Type":"ContainerDied","Data":"830f52ed8d9cc635ab2bb9761dfde1820ca1b7aa4befb2d469b83b430fd81421"} Jul 14 00:59:46.738657 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.738335 2557 scope.go:117] "RemoveContainer" containerID="886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86" Jul 14 00:59:46.748624 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.748606 2557 scope.go:117] "RemoveContainer" containerID="886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86" Jul 14 00:59:46.748925 ip-10-0-134-143 kubenswrapper[2557]: E0714 00:59:46.748900 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86\": container with ID starting with 886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86 not found: ID does not exist" containerID="886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86" Jul 14 00:59:46.749017 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.748931 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86"} err="failed to get container status \"886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86\": rpc error: code = NotFound desc = could not find container \"886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86\": container with ID starting with 886964f716427ccadcd2b471db339c9c5050d345a7d07d966cb6b5530257ee86 not found: ID does not exist" Jul 14 00:59:46.767981 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.767952 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6c6d4fdd8c-st2tp"] Jul 14 00:59:46.774176 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:46.774140 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6c6d4fdd8c-st2tp"] Jul 14 00:59:47.203647 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:47.203603 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 00:59:47.678533 ip-10-0-134-143 kubenswrapper[2557]: I0714 00:59:47.678497 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c59e793c-c62e-4577-a87d-7f8adc429add" path="/var/lib/kubelet/pods/c59e793c-c62e-4577-a87d-7f8adc429add/volumes" Jul 14 01:00:28.235125 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:00:28.235083 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:00:43.408847 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:00:43.408812 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:00:43.458750 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:00:43.458722 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:00:56.034041 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:00:56.033925 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:00:57.912181 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:00:57.912089 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:01:26.336903 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:01:26.336856 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:01:39.449936 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:01:39.449905 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:01:39.549707 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:01:39.549677 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:01:39.664126 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:01:39.664069 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:01:45.693450 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:01:45.693411 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:01:46.174075 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:01:46.174041 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:01:46.264214 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:01:46.264182 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:01:51.856434 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:01:51.856384 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:01:51.958777 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:01:51.958741 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:02:29.387417 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:02:29.387381 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:03:12.125286 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:03:12.125251 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:04:23.620545 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:04:23.620511 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:04:23.627016 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:04:23.626767 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:05:47.942875 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:47.942840 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t"] Jul 14 01:05:47.943419 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:47.943042 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" podUID="36d2e455-dbc5-4663-b70b-f1e8ec0dde97" containerName="manager" containerID="cri-o://f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795" gracePeriod=10 Jul 14 01:05:51.488019 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.487995 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 01:05:51.566788 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.566759 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mjdf\" (UniqueName: \"kubernetes.io/projected/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-kube-api-access-2mjdf\") pod \"36d2e455-dbc5-4663-b70b-f1e8ec0dde97\" (UID: \"36d2e455-dbc5-4663-b70b-f1e8ec0dde97\") " Jul 14 01:05:51.566935 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.566871 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-extensions-socket-volume\") pod \"36d2e455-dbc5-4663-b70b-f1e8ec0dde97\" (UID: \"36d2e455-dbc5-4663-b70b-f1e8ec0dde97\") " Jul 14 01:05:51.567272 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.567249 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "36d2e455-dbc5-4663-b70b-f1e8ec0dde97" (UID: "36d2e455-dbc5-4663-b70b-f1e8ec0dde97"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:05:51.568679 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.568655 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-kube-api-access-2mjdf" (OuterVolumeSpecName: "kube-api-access-2mjdf") pod "36d2e455-dbc5-4663-b70b-f1e8ec0dde97" (UID: "36d2e455-dbc5-4663-b70b-f1e8ec0dde97"). InnerVolumeSpecName "kube-api-access-2mjdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:05:51.667918 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.667855 2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-extensions-socket-volume\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:05:51.667918 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.667879 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mjdf\" (UniqueName: \"kubernetes.io/projected/36d2e455-dbc5-4663-b70b-f1e8ec0dde97-kube-api-access-2mjdf\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:05:51.687430 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.687404 2557 generic.go:356] "Generic (PLEG): container finished" podID="36d2e455-dbc5-4663-b70b-f1e8ec0dde97" containerID="f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795" exitCode=0 Jul 14 01:05:51.687522 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.687448 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" Jul 14 01:05:51.687522 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.687485 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" event={"ID":"36d2e455-dbc5-4663-b70b-f1e8ec0dde97","Type":"ContainerDied","Data":"f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795"} Jul 14 01:05:51.687592 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.687533 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t" event={"ID":"36d2e455-dbc5-4663-b70b-f1e8ec0dde97","Type":"ContainerDied","Data":"215e993f3fbb3624c83088bd4fdbaab4301f4446a9d0ec9f8f4c91946db22064"} Jul 14 01:05:51.687592 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.687555 2557 scope.go:117] "RemoveContainer" containerID="f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795" Jul 14 01:05:51.697786 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.697770 2557 scope.go:117] "RemoveContainer" containerID="f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795" Jul 14 01:05:51.698047 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:05:51.698029 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795\": container with ID starting with f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795 not found: ID does not exist" containerID="f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795" Jul 14 01:05:51.698110 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.698069 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795"} err="failed to get container status \"f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795\": rpc error: code = NotFound desc = could not find container \"f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795\": container with ID starting with f7b4c4fc0953b18094f1319c66d448c26916145a6ce8144e9ab1beca8f7b2795 not found: ID does not exist" Jul 14 01:05:51.701969 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.701946 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t"] Jul 14 01:05:51.707301 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:51.707281 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rj78t"] Jul 14 01:05:53.677752 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:05:53.677717 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36d2e455-dbc5-4663-b70b-f1e8ec0dde97" path="/var/lib/kubelet/pods/36d2e455-dbc5-4663-b70b-f1e8ec0dde97/volumes" Jul 14 01:06:49.066509 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.066429 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n"] Jul 14 01:06:49.067248 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.066858 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c59e793c-c62e-4577-a87d-7f8adc429add" containerName="authorino" Jul 14 01:06:49.067248 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.066872 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c59e793c-c62e-4577-a87d-7f8adc429add" containerName="authorino" Jul 14 01:06:49.067248 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.066883 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36d2e455-dbc5-4663-b70b-f1e8ec0dde97" containerName="manager" Jul 14 01:06:49.067248 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.066888 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="36d2e455-dbc5-4663-b70b-f1e8ec0dde97" containerName="manager" Jul 14 01:06:49.067248 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.066945 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="36d2e455-dbc5-4663-b70b-f1e8ec0dde97" containerName="manager" Jul 14 01:06:49.067248 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.066955 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c59e793c-c62e-4577-a87d-7f8adc429add" containerName="authorino" Jul 14 01:06:49.071207 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.071191 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" Jul 14 01:06:49.073237 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.073213 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-nxppt\"" Jul 14 01:06:49.082827 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.082806 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n"] Jul 14 01:06:49.124802 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.124770 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmhxs\" (UniqueName: \"kubernetes.io/projected/5944e5ac-0819-45fa-b8c3-871a0a110e34-kube-api-access-vmhxs\") pod \"kuadrant-operator-controller-manager-86878c769f-swm4n\" (UID: \"5944e5ac-0819-45fa-b8c3-871a0a110e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" Jul 14 01:06:49.124918 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.124847 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5944e5ac-0819-45fa-b8c3-871a0a110e34-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-swm4n\" (UID: \"5944e5ac-0819-45fa-b8c3-871a0a110e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" Jul 14 01:06:49.225373 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.225347 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmhxs\" (UniqueName: \"kubernetes.io/projected/5944e5ac-0819-45fa-b8c3-871a0a110e34-kube-api-access-vmhxs\") pod \"kuadrant-operator-controller-manager-86878c769f-swm4n\" (UID: \"5944e5ac-0819-45fa-b8c3-871a0a110e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" Jul 14 01:06:49.225500 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.225414 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5944e5ac-0819-45fa-b8c3-871a0a110e34-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-swm4n\" (UID: \"5944e5ac-0819-45fa-b8c3-871a0a110e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" Jul 14 01:06:49.225767 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.225751 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5944e5ac-0819-45fa-b8c3-871a0a110e34-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-swm4n\" (UID: \"5944e5ac-0819-45fa-b8c3-871a0a110e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" Jul 14 01:06:49.232877 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.232853 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmhxs\" (UniqueName: \"kubernetes.io/projected/5944e5ac-0819-45fa-b8c3-871a0a110e34-kube-api-access-vmhxs\") pod \"kuadrant-operator-controller-manager-86878c769f-swm4n\" (UID: \"5944e5ac-0819-45fa-b8c3-871a0a110e34\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" Jul 14 01:06:49.381461 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.381406 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" Jul 14 01:06:49.518365 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:49.518343 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n"] Jul 14 01:06:49.520608 ip-10-0-134-143 kubenswrapper[2557]: W0714 01:06:49.520586 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5944e5ac_0819_45fa_b8c3_871a0a110e34.slice/crio-c57757aeb2e339f023a48cce67b284f1686470672c722186fcd9a5a7a6ba1260 WatchSource:0}: Error finding container c57757aeb2e339f023a48cce67b284f1686470672c722186fcd9a5a7a6ba1260: Status 404 returned error can't find the container with id c57757aeb2e339f023a48cce67b284f1686470672c722186fcd9a5a7a6ba1260 Jul 14 01:06:50.002193 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:50.002128 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" event={"ID":"5944e5ac-0819-45fa-b8c3-871a0a110e34","Type":"ContainerStarted","Data":"263c63b9e7f5f1e4845488d7a46773105a06047a16d8548f87530404bdedd46f"} Jul 14 01:06:50.002383 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:50.002211 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" Jul 14 01:06:50.002383 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:50.002229 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" event={"ID":"5944e5ac-0819-45fa-b8c3-871a0a110e34","Type":"ContainerStarted","Data":"c57757aeb2e339f023a48cce67b284f1686470672c722186fcd9a5a7a6ba1260"} Jul 14 01:06:50.024493 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:06:50.024453 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" podStartSLOduration=1.024437618 podStartE2EDuration="1.024437618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:06:50.022114048 +0000 UTC m=+1346.873481417" watchObservedRunningTime="2026-07-14 01:06:50.024437618 +0000 UTC m=+1346.875805047" Jul 14 01:07:01.009282 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:07:01.009253 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-swm4n" Jul 14 01:07:28.333176 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:07:28.333115 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:07:33.258266 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:07:33.258222 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:07:36.273768 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:07:36.273720 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:07:36.375547 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:07:36.375514 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:07:49.621002 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:07:49.620962 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:08:42.891051 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:08:42.891006 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:08:53.727733 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:08:53.727672 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:09:23.662850 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:09:23.662818 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:09:23.671475 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:09:23.671455 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:12:02.957956 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:12:02.957919 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:12:03.045595 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:12:03.045563 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:12:38.324368 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:12:38.324324 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:12:45.782351 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:12:45.782251 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:12:54.069123 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:12:54.069087 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:12:59.175009 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:12:59.174971 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:02.179215 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:02.179179 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:02.279660 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:02.279621 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:02.640672 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:02.640636 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:08.491884 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.491849 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:08.548007 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.547976 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:08.675620 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.675592 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s"] Jul 14 01:13:08.679382 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.679360 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.681677 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.681644 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b9a4f161-openshift-default-dockercfg-9fjvg\"" Jul 14 01:13:08.703028 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.703001 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s"] Jul 14 01:13:08.774621 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.774540 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-envoy\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.774773 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.774695 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93936539-40e9-4d03-8424-edac2323f749-istiod-ca-cert\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.774773 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.774759 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-istio-token\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.774904 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.774787 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-certs\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.774963 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.774925 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-credential-socket\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.775020 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.774986 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgdrw\" (UniqueName: \"kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-kube-api-access-cgdrw\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.775081 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.775050 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-data\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.775140 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.775086 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93936539-40e9-4d03-8424-edac2323f749-istio-podinfo\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.775210 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.775139 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-socket\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.876458 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.876425 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-socket\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.876754 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.876733 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-envoy\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.876892 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.876877 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93936539-40e9-4d03-8424-edac2323f749-istiod-ca-cert\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.877031 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.877016 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-istio-token\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.877130 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.877117 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-certs\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.877282 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.877267 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-credential-socket\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.877391 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.877378 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgdrw\" (UniqueName: \"kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-kube-api-access-cgdrw\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.877519 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.877504 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-data\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.877625 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.877611 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93936539-40e9-4d03-8424-edac2323f749-istio-podinfo\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.878006 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.877978 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-socket\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.879480 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.878792 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93936539-40e9-4d03-8424-edac2323f749-istiod-ca-cert\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.879480 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.878938 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-credential-socket\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.879480 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.879421 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-data\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.883812 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.879836 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-certs\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.884053 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.884030 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-envoy\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.885839 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.885818 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93936539-40e9-4d03-8424-edac2323f749-istio-podinfo\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.889661 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.889625 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:08.893994 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.893967 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-istio-token\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.894126 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.894109 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgdrw\" (UniqueName: \"kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-kube-api-access-cgdrw\") pod \"e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:08.963080 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.963054 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:08.990950 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:08.990926 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:09.132601 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:09.132578 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s"] Jul 14 01:13:09.134443 ip-10-0-134-143 kubenswrapper[2557]: W0714 01:13:09.134419 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93936539_40e9_4d03_8424_edac2323f749.slice/crio-09ffa56cb2d40790136da5a9c3cb7ed57e23fb0fafbc6fdf81b1c798bd8e94d8 WatchSource:0}: Error finding container 09ffa56cb2d40790136da5a9c3cb7ed57e23fb0fafbc6fdf81b1c798bd8e94d8: Status 404 returned error can't find the container with id 09ffa56cb2d40790136da5a9c3cb7ed57e23fb0fafbc6fdf81b1c798bd8e94d8 Jul 14 01:13:09.136386 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:09.136353 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:13:09.136474 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:09.136420 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:13:09.136474 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:09.136448 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:13:09.879027 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:09.878995 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:09.976167 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:09.976129 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:10.054415 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:10.054381 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" event={"ID":"93936539-40e9-4d03-8424-edac2323f749","Type":"ContainerStarted","Data":"9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f"} Jul 14 01:13:10.054590 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:10.054441 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" event={"ID":"93936539-40e9-4d03-8424-edac2323f749","Type":"ContainerStarted","Data":"09ffa56cb2d40790136da5a9c3cb7ed57e23fb0fafbc6fdf81b1c798bd8e94d8"} Jul 14 01:13:10.075640 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:10.075584 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" podStartSLOduration=2.075565441 podStartE2EDuration="2.075565441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:13:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:13:10.071049323 +0000 UTC m=+1726.922416693" watchObservedRunningTime="2026-07-14 01:13:10.075565441 +0000 UTC m=+1726.926932812" Jul 14 01:13:10.136288 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:10.136203 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:10.635815 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:10.635774 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:10.760574 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:10.760509 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:10.860807 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:10.860773 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:10.991231 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:10.991135 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:10.996113 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:10.996091 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:11.059901 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:11.059873 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:11.060765 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:11.060745 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:12.455074 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:12.455037 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:13:24.141784 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:24.141725 2557 generic.go:356] "Generic (PLEG): container finished" podID="93936539-40e9-4d03-8424-edac2323f749" containerID="9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f" exitCode=0 Jul 14 01:13:24.142110 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:24.141783 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" event={"ID":"93936539-40e9-4d03-8424-edac2323f749","Type":"ContainerDied","Data":"9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f"} Jul 14 01:13:24.142213 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:24.142195 2557 scope.go:117] "RemoveContainer" containerID="9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f" Jul 14 01:13:24.143168 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:24.143127 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:13:24.143233 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:24.143221 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:13:24.143277 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:24.143262 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:13:25.151723 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:25.151685 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" event={"ID":"93936539-40e9-4d03-8424-edac2323f749","Type":"ContainerStarted","Data":"16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974"} Jul 14 01:13:25.992910 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:25.992868 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:25.993124 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:25.993090 2557 patch_prober.go:28] interesting pod/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 14 01:13:25.993225 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:25.993170 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 14 01:13:26.060068 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:26.060045 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:26.991833 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:26.991795 2557 patch_prober.go:28] interesting pod/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 14 01:13:26.992399 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:26.992366 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 14 01:13:27.992321 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:27.992280 2557 patch_prober.go:28] interesting pod/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 14 01:13:27.992818 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:27.992341 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 14 01:13:28.991699 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:28.991664 2557 patch_prober.go:28] interesting pod/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body= Jul 14 01:13:28.991993 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:28.991965 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" Jul 14 01:13:29.995698 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:29.995666 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:13:29.996661 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:13:29.996643 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:14:23.708687 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:23.708658 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:14:23.716667 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:23.716647 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:14:25.382491 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:25.382452 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:30.018458 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:30.018422 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:35.653411 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:35.653381 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:35.719444 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:35.719408 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:40.191223 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:40.191179 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s"] Jul 14 01:14:40.191649 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:40.191407 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" containerID="cri-o://16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974" gracePeriod=30 Jul 14 01:14:40.216354 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:40.216326 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:40.590884 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:40.590849 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:40.695776 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:40.695744 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:40.790662 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:40.790633 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:42.040645 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:42.040609 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:42.196006 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:42.195969 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:42.636566 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:42.636530 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:42.737200 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:42.737142 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:43.091062 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:43.091030 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:44.998455 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:44.998409 2557 patch_prober.go:28] interesting pod/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 01:14:44.998895 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:44.998481 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 01:14:45.458781 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.458758 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:14:45.568995 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.568934 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-envoy\") pod \"93936539-40e9-4d03-8424-edac2323f749\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " Jul 14 01:14:45.568995 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.568981 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-data\") pod \"93936539-40e9-4d03-8424-edac2323f749\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " Jul 14 01:14:45.569200 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569004 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-istio-token\") pod \"93936539-40e9-4d03-8424-edac2323f749\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " Jul 14 01:14:45.569200 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569036 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-credential-socket\") pod \"93936539-40e9-4d03-8424-edac2323f749\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " Jul 14 01:14:45.569200 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569064 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93936539-40e9-4d03-8424-edac2323f749-istiod-ca-cert\") pod \"93936539-40e9-4d03-8424-edac2323f749\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " Jul 14 01:14:45.569200 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569079 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgdrw\" (UniqueName: \"kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-kube-api-access-cgdrw\") pod \"93936539-40e9-4d03-8424-edac2323f749\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " Jul 14 01:14:45.569200 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569102 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-socket\") pod \"93936539-40e9-4d03-8424-edac2323f749\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " Jul 14 01:14:45.569200 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569139 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93936539-40e9-4d03-8424-edac2323f749-istio-podinfo\") pod \"93936539-40e9-4d03-8424-edac2323f749\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " Jul 14 01:14:45.569533 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569238 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-certs\") pod \"93936539-40e9-4d03-8424-edac2323f749\" (UID: \"93936539-40e9-4d03-8424-edac2323f749\") " Jul 14 01:14:45.569533 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569304 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "93936539-40e9-4d03-8424-edac2323f749" (UID: "93936539-40e9-4d03-8424-edac2323f749"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:14:45.569533 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569482 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "93936539-40e9-4d03-8424-edac2323f749" (UID: "93936539-40e9-4d03-8424-edac2323f749"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:14:45.569533 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569500 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93936539-40e9-4d03-8424-edac2323f749-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "93936539-40e9-4d03-8424-edac2323f749" (UID: "93936539-40e9-4d03-8424-edac2323f749"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:14:45.569739 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569620 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-credential-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:14:45.569739 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569640 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/93936539-40e9-4d03-8424-edac2323f749-istiod-ca-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:14:45.569739 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569655 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-certs\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:14:45.569739 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569657 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "93936539-40e9-4d03-8424-edac2323f749" (UID: "93936539-40e9-4d03-8424-edac2323f749"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:14:45.569953 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.569918 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-data" (OuterVolumeSpecName: "istio-data") pod "93936539-40e9-4d03-8424-edac2323f749" (UID: "93936539-40e9-4d03-8424-edac2323f749"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:14:45.571329 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.571296 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-istio-token" (OuterVolumeSpecName: "istio-token") pod "93936539-40e9-4d03-8424-edac2323f749" (UID: "93936539-40e9-4d03-8424-edac2323f749"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:14:45.571571 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.571550 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "93936539-40e9-4d03-8424-edac2323f749" (UID: "93936539-40e9-4d03-8424-edac2323f749"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:14:45.571666 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.571562 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/93936539-40e9-4d03-8424-edac2323f749-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "93936539-40e9-4d03-8424-edac2323f749" (UID: "93936539-40e9-4d03-8424-edac2323f749"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:14:45.571815 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.571787 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-kube-api-access-cgdrw" (OuterVolumeSpecName: "kube-api-access-cgdrw") pod "93936539-40e9-4d03-8424-edac2323f749" (UID: "93936539-40e9-4d03-8424-edac2323f749"). InnerVolumeSpecName "kube-api-access-cgdrw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:14:45.619914 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.619887 2557 generic.go:356] "Generic (PLEG): container finished" podID="93936539-40e9-4d03-8424-edac2323f749" containerID="16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974" exitCode=0 Jul 14 01:14:45.620039 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.619934 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" Jul 14 01:14:45.620039 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.619960 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" event={"ID":"93936539-40e9-4d03-8424-edac2323f749","Type":"ContainerDied","Data":"16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974"} Jul 14 01:14:45.620039 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.620003 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s" event={"ID":"93936539-40e9-4d03-8424-edac2323f749","Type":"ContainerDied","Data":"09ffa56cb2d40790136da5a9c3cb7ed57e23fb0fafbc6fdf81b1c798bd8e94d8"} Jul 14 01:14:45.620039 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.620023 2557 scope.go:117] "RemoveContainer" containerID="16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974" Jul 14 01:14:45.629707 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.629691 2557 scope.go:117] "RemoveContainer" containerID="9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f" Jul 14 01:14:45.637793 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.637775 2557 scope.go:117] "RemoveContainer" containerID="16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974" Jul 14 01:14:45.638052 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:14:45.638032 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974\": container with ID starting with 16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974 not found: ID does not exist" containerID="16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974" Jul 14 01:14:45.638217 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.638062 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974"} err="failed to get container status \"16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974\": rpc error: code = NotFound desc = could not find container \"16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974\": container with ID starting with 16d1e1ed168988d2f999504dfc343b053492b71ddcce7f78dbf4640327fcf974 not found: ID does not exist" Jul 14 01:14:45.638217 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.638086 2557 scope.go:117] "RemoveContainer" containerID="9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f" Jul 14 01:14:45.638344 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:14:45.638327 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f\": container with ID starting with 9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f not found: ID does not exist" containerID="9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f" Jul 14 01:14:45.638386 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.638350 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f"} err="failed to get container status \"9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f\": rpc error: code = NotFound desc = could not find container \"9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f\": container with ID starting with 9c610959cff77bbda71dbab88f99647764ca7de8e7a698f23772e2ac5889082f not found: ID does not exist" Jul 14 01:14:45.642404 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.642376 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s"] Jul 14 01:14:45.646001 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.645981 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b9a4f161-openshift-default-85bb68d447-j7c6s"] Jul 14 01:14:45.670129 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.670111 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-envoy\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:14:45.670129 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.670130 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-istio-data\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:14:45.670267 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.670138 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-istio-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:14:45.670267 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.670172 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgdrw\" (UniqueName: \"kubernetes.io/projected/93936539-40e9-4d03-8424-edac2323f749-kube-api-access-cgdrw\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:14:45.670267 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.670186 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/93936539-40e9-4d03-8424-edac2323f749-workload-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:14:45.670267 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.670200 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/93936539-40e9-4d03-8424-edac2323f749-istio-podinfo\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:14:45.678467 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:45.678442 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93936539-40e9-4d03-8424-edac2323f749" path="/var/lib/kubelet/pods/93936539-40e9-4d03-8424-edac2323f749/volumes" Jul 14 01:14:46.668897 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:46.668850 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:46.769133 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:46.769094 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:50.065733 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:50.065693 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:57.306880 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:57.306812 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:14:57.360717 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:14:57.360678 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:46.702378 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:46.702289 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:46.817911 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:46.817862 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:51.885428 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:51.885391 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:52.825789 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:52.825750 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:52.965561 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:52.965511 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:58.281084 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.281055 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:58.339304 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.339267 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:58.464323 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.464293 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s"] Jul 14 01:15:58.464903 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.464872 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" Jul 14 01:15:58.465053 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.464956 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" Jul 14 01:15:58.465185 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.465143 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" Jul 14 01:15:58.465281 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.465193 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" Jul 14 01:15:58.468271 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.468245 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.470394 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.470367 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f0fe996a-openshift-default-dockercfg-8bgft\"" Jul 14 01:15:58.484441 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.484415 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s"] Jul 14 01:15:58.504933 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.504908 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:58.560398 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.560371 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:58.655869 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.655839 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-socket\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.656015 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.655875 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-certs\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.656015 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.655912 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-podinfo\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.656015 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.655954 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-token\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.656015 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.655977 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e0c70a-dc62-4876-9936-23655a6fb70e-istiod-ca-cert\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.656015 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.655995 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-envoy\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.656015 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.656010 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-data\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.656254 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.656049 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-credential-socket\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.656254 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.656139 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gqln\" (UniqueName: \"kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-kube-api-access-5gqln\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757047 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757023 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-podinfo\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757192 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757059 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-token\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757192 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757083 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e0c70a-dc62-4876-9936-23655a6fb70e-istiod-ca-cert\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757192 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757105 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-envoy\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757317 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757223 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-data\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757317 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757288 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-credential-socket\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757435 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757340 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqln\" (UniqueName: \"kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-kube-api-access-5gqln\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757504 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757452 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-socket\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757563 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757498 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-certs\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757672 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757628 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-data\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757800 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757700 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-credential-socket\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757867 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757833 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-certs\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757928 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757868 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-socket\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.757928 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.757918 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e0c70a-dc62-4876-9936-23655a6fb70e-istiod-ca-cert\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.759383 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.759364 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-envoy\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.759646 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.759628 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-podinfo\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.764012 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.763987 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-token\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.764205 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.764188 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gqln\" (UniqueName: \"kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-kube-api-access-5gqln\") pod \"e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.783704 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.783687 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:58.921257 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.921231 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s"] Jul 14 01:15:58.922786 ip-10-0-134-143 kubenswrapper[2557]: W0714 01:15:58.922761 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50e0c70a_dc62_4876_9936_23655a6fb70e.slice/crio-d5bb5f7019f523698cc7c314269fe94ce721ee69debc304cf00390397455a688 WatchSource:0}: Error finding container d5bb5f7019f523698cc7c314269fe94ce721ee69debc304cf00390397455a688: Status 404 returned error can't find the container with id d5bb5f7019f523698cc7c314269fe94ce721ee69debc304cf00390397455a688 Jul 14 01:15:58.924859 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.924825 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:15:58.924920 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.924887 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:15:58.924920 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:58.924915 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:15:59.026289 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:59.026242 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" event={"ID":"50e0c70a-dc62-4876-9936-23655a6fb70e","Type":"ContainerStarted","Data":"b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb"} Jul 14 01:15:59.026456 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:59.026306 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" event={"ID":"50e0c70a-dc62-4876-9936-23655a6fb70e","Type":"ContainerStarted","Data":"d5bb5f7019f523698cc7c314269fe94ce721ee69debc304cf00390397455a688"} Jul 14 01:15:59.043388 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:59.043337 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" podStartSLOduration=1.043318335 podStartE2EDuration="1.043318335s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:15:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:15:59.042070774 +0000 UTC m=+1895.893438145" watchObservedRunningTime="2026-07-14 01:15:59.043318335 +0000 UTC m=+1895.894685706" Jul 14 01:15:59.613199 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:59.613142 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:59.687955 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:59.687913 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:59.777973 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:59.777930 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:15:59.784747 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:59.784724 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:59.790655 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:59.790623 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:15:59.988423 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:15:59.988333 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:16:00.031953 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:00.031924 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:16:00.032967 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:00.032948 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:16:00.159775 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:00.159742 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:16:00.269380 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:00.269296 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:16:01.772286 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:01.772252 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:16:11.092862 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:11.092830 2557 generic.go:356] "Generic (PLEG): container finished" podID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerID="b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb" exitCode=0 Jul 14 01:16:11.093285 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:11.092906 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" event={"ID":"50e0c70a-dc62-4876-9936-23655a6fb70e","Type":"ContainerDied","Data":"b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb"} Jul 14 01:16:11.093285 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:11.093275 2557 scope.go:117] "RemoveContainer" containerID="b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb" Jul 14 01:16:11.094205 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:11.094181 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:16:11.094280 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:11.094242 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:16:11.094321 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:11.094280 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:16:12.108726 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:12.108692 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" event={"ID":"50e0c70a-dc62-4876-9936-23655a6fb70e","Type":"ContainerStarted","Data":"a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2"} Jul 14 01:16:12.784808 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:12.784773 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:16:12.786100 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:12.786069 2557 patch_prober.go:28] interesting pod/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 14 01:16:12.786253 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:12.786128 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 14 01:16:13.785019 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:13.784982 2557 patch_prober.go:28] interesting pod/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 14 01:16:13.786024 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:13.785620 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 14 01:16:14.784755 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:14.784717 2557 patch_prober.go:28] interesting pod/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 14 01:16:14.785058 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:14.785027 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 14 01:16:15.032517 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:15.032477 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:16:15.784087 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:15.784036 2557 patch_prober.go:28] interesting pod/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body= Jul 14 01:16:15.784320 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:15.784105 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" Jul 14 01:16:16.787892 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:16.787861 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:16:16.788665 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:16.788644 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:16:20.417924 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:20.417883 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:16:20.595481 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:20.595445 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:16:21.143852 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:21.143821 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:16:21.244940 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:21.244905 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:16:34.648969 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:16:34.648916 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:05.245388 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:05.245341 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:09.892737 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:09.892705 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:15.837166 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:15.837111 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:15.938519 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:15.938492 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:20.952920 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:20.952881 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s"] Jul 14 01:17:20.953323 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:20.953171 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerName="istio-proxy" containerID="cri-o://a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2" gracePeriod=30 Jul 14 01:17:20.974802 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:20.974770 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:21.284353 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:21.284311 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:22.398411 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:22.398382 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:22.498689 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:22.498660 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:22.630499 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:22.630467 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:22.912664 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:22.912619 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:23.245584 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:23.245494 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:23.347423 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:23.347385 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:26.218751 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.218698 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:17:26.257130 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257100 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-token\") pod \"50e0c70a-dc62-4876-9936-23655a6fb70e\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " Jul 14 01:17:26.257300 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257176 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-socket\") pod \"50e0c70a-dc62-4876-9936-23655a6fb70e\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " Jul 14 01:17:26.257869 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257436 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e0c70a-dc62-4876-9936-23655a6fb70e-istiod-ca-cert\") pod \"50e0c70a-dc62-4876-9936-23655a6fb70e\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " Jul 14 01:17:26.257869 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257448 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "50e0c70a-dc62-4876-9936-23655a6fb70e" (UID: "50e0c70a-dc62-4876-9936-23655a6fb70e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:17:26.257869 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257493 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-credential-socket\") pod \"50e0c70a-dc62-4876-9936-23655a6fb70e\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " Jul 14 01:17:26.257869 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257525 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-envoy\") pod \"50e0c70a-dc62-4876-9936-23655a6fb70e\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " Jul 14 01:17:26.257869 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257551 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gqln\" (UniqueName: \"kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-kube-api-access-5gqln\") pod \"50e0c70a-dc62-4876-9936-23655a6fb70e\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " Jul 14 01:17:26.257869 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257602 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-data\") pod \"50e0c70a-dc62-4876-9936-23655a6fb70e\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " Jul 14 01:17:26.257869 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257629 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-certs\") pod \"50e0c70a-dc62-4876-9936-23655a6fb70e\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " Jul 14 01:17:26.257869 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257687 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-podinfo\") pod \"50e0c70a-dc62-4876-9936-23655a6fb70e\" (UID: \"50e0c70a-dc62-4876-9936-23655a6fb70e\") " Jul 14 01:17:26.257869 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257743 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "50e0c70a-dc62-4876-9936-23655a6fb70e" (UID: "50e0c70a-dc62-4876-9936-23655a6fb70e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:17:26.258425 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.257856 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50e0c70a-dc62-4876-9936-23655a6fb70e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "50e0c70a-dc62-4876-9936-23655a6fb70e" (UID: "50e0c70a-dc62-4876-9936-23655a6fb70e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:17:26.258425 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.258044 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-credential-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:17:26.258425 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.258067 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:17:26.258425 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.258081 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50e0c70a-dc62-4876-9936-23655a6fb70e-istiod-ca-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:17:26.258914 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.258805 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "50e0c70a-dc62-4876-9936-23655a6fb70e" (UID: "50e0c70a-dc62-4876-9936-23655a6fb70e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:17:26.259083 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.259060 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-data" (OuterVolumeSpecName: "istio-data") pod "50e0c70a-dc62-4876-9936-23655a6fb70e" (UID: "50e0c70a-dc62-4876-9936-23655a6fb70e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:17:26.259614 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.259584 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-token" (OuterVolumeSpecName: "istio-token") pod "50e0c70a-dc62-4876-9936-23655a6fb70e" (UID: "50e0c70a-dc62-4876-9936-23655a6fb70e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:17:26.260312 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.260285 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-kube-api-access-5gqln" (OuterVolumeSpecName: "kube-api-access-5gqln") pod "50e0c70a-dc62-4876-9936-23655a6fb70e" (UID: "50e0c70a-dc62-4876-9936-23655a6fb70e"). InnerVolumeSpecName "kube-api-access-5gqln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:17:26.260900 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.260864 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "50e0c70a-dc62-4876-9936-23655a6fb70e" (UID: "50e0c70a-dc62-4876-9936-23655a6fb70e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:17:26.261564 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.261536 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "50e0c70a-dc62-4876-9936-23655a6fb70e" (UID: "50e0c70a-dc62-4876-9936-23655a6fb70e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:17:26.359382 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.359356 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:17:26.359382 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.359379 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-envoy\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:17:26.359522 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.359390 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gqln\" (UniqueName: \"kubernetes.io/projected/50e0c70a-dc62-4876-9936-23655a6fb70e-kube-api-access-5gqln\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:17:26.359522 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.359399 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-data\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:17:26.359522 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.359407 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50e0c70a-dc62-4876-9936-23655a6fb70e-workload-certs\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:17:26.359522 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.359415 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50e0c70a-dc62-4876-9936-23655a6fb70e-istio-podinfo\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:17:26.518341 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.518277 2557 generic.go:356] "Generic (PLEG): container finished" podID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerID="a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2" exitCode=0 Jul 14 01:17:26.518341 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.518318 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" event={"ID":"50e0c70a-dc62-4876-9936-23655a6fb70e","Type":"ContainerDied","Data":"a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2"} Jul 14 01:17:26.518524 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.518359 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" event={"ID":"50e0c70a-dc62-4876-9936-23655a6fb70e","Type":"ContainerDied","Data":"d5bb5f7019f523698cc7c314269fe94ce721ee69debc304cf00390397455a688"} Jul 14 01:17:26.518524 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.518380 2557 scope.go:117] "RemoveContainer" containerID="a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2" Jul 14 01:17:26.518524 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.518389 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s" Jul 14 01:17:26.529543 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.529525 2557 scope.go:117] "RemoveContainer" containerID="b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb" Jul 14 01:17:26.538123 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.538104 2557 scope.go:117] "RemoveContainer" containerID="a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2" Jul 14 01:17:26.538428 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:17:26.538404 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2\": container with ID starting with a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2 not found: ID does not exist" containerID="a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2" Jul 14 01:17:26.538485 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.538452 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2"} err="failed to get container status \"a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2\": rpc error: code = NotFound desc = could not find container \"a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2\": container with ID starting with a7b911dd78bab5e616dc1d408b4cf184dfea09ed6ccac6621b777e3239d875e2 not found: ID does not exist" Jul 14 01:17:26.538485 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.538477 2557 scope.go:117] "RemoveContainer" containerID="b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb" Jul 14 01:17:26.538770 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:17:26.538752 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb\": container with ID starting with b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb not found: ID does not exist" containerID="b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb" Jul 14 01:17:26.538821 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.538778 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb"} err="failed to get container status \"b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb\": rpc error: code = NotFound desc = could not find container \"b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb\": container with ID starting with b061769453e2df7c084a52b7b3d9b9afb6a21f73cf622209683d04a303484ebb not found: ID does not exist" Jul 14 01:17:26.539846 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.539827 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s"] Jul 14 01:17:26.543698 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:26.543678 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f0fe996a-openshift-default-7f9f99ddf5-qhm8s"] Jul 14 01:17:27.680310 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:27.680222 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" path="/var/lib/kubelet/pods/50e0c70a-dc62-4876-9936-23655a6fb70e/volumes" Jul 14 01:17:27.806777 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:27.806745 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:27.861697 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:27.861652 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:17:31.011626 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:17:31.011590 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:27.441996 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:27.441947 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:31.946425 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:31.946393 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:37.558667 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:37.558596 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:37.659125 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:37.659093 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:42.186024 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:42.185984 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:42.817570 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:42.817536 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:42.964265 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:42.964231 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:44.553620 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:44.553576 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:49.778491 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:49.778455 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:50.426473 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:50.426440 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:50.491901 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:50.491870 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:50.569865 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:50.569816 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:50.669130 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:50.669100 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:50.768792 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:50.768717 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:18:51.750758 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:18:51.750719 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:19:17.128955 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:17.128911 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:19:22.745495 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:22.745465 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:19:22.840364 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:22.840335 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:19:23.719293 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:23.719252 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:19:23.741982 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:23.741955 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:19:23.750796 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:23.750776 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:19:38.809676 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:38.809635 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:19:44.349036 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:44.348998 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:19:44.408020 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:44.407989 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:19:44.490205 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:44.490176 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:19:44.589115 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:44.589087 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:19:55.070941 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:55.070903 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:19:56.074850 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:19:56.074808 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:01.763305 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:01.763270 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:01.865394 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:01.865353 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:01.957791 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:01.957759 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:02.057933 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:02.057906 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:02.818618 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:02.818580 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:02.917990 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:02.917949 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:03.018068 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:03.018038 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:04.670363 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:04.670328 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:10.143693 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:10.143651 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:10.205847 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:10.205816 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:10.806502 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:10.806473 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:16.398677 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.398638 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:16.455985 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.455940 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:16.631146 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.631110 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:16.918542 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.918509 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r"] Jul 14 01:20:16.919066 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.919050 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerName="istio-proxy" Jul 14 01:20:16.919114 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.919070 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerName="istio-proxy" Jul 14 01:20:16.919114 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.919091 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerName="istio-proxy" Jul 14 01:20:16.919114 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.919100 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerName="istio-proxy" Jul 14 01:20:16.919227 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.919132 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" Jul 14 01:20:16.919227 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.919142 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="93936539-40e9-4d03-8424-edac2323f749" containerName="istio-proxy" Jul 14 01:20:16.919289 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.919237 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerName="istio-proxy" Jul 14 01:20:16.919289 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.919251 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="50e0c70a-dc62-4876-9936-23655a6fb70e" containerName="istio-proxy" Jul 14 01:20:16.922546 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.922529 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:16.924415 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.924389 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2a62f7c8-openshift-default-dockercfg-z7jn9\"" Jul 14 01:20:16.932117 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.932092 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r"] Jul 14 01:20:16.943418 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:16.943384 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:17.009061 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.009035 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-podinfo\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.009214 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.009071 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-token\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.009214 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.009098 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfd2w\" (UniqueName: \"kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-kube-api-access-sfd2w\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.009346 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.009234 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-credential-socket\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.009346 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.009280 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-envoy\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.009464 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.009353 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-socket\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.009464 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.009380 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-certs\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.009464 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.009405 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a005e203-ba4b-40b4-89ed-e2e661b38e42-istiod-ca-cert\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.009464 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.009434 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-data\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.110504 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.110471 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-socket\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.110636 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.110516 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-certs\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.110636 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.110541 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a005e203-ba4b-40b4-89ed-e2e661b38e42-istiod-ca-cert\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.110636 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.110569 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-data\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.110636 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.110623 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-podinfo\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.110835 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.110663 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-token\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.110835 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.110701 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfd2w\" (UniqueName: \"kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-kube-api-access-sfd2w\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.110835 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.110756 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-credential-socket\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.110835 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.110778 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-envoy\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.111051 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.110960 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-socket\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.111051 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.111015 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-certs\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.111295 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.111274 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-credential-socket\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.111406 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.111381 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a005e203-ba4b-40b4-89ed-e2e661b38e42-istiod-ca-cert\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.111490 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.111480 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-data\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.113211 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.113184 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-envoy\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.113376 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.113356 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-podinfo\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.117894 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.117874 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-token\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.118071 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.118051 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfd2w\" (UniqueName: \"kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-kube-api-access-sfd2w\") pod \"e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.233985 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.233920 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:17.312261 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.312126 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:17.396362 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.396332 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r"] Jul 14 01:20:17.397091 ip-10-0-134-143 kubenswrapper[2557]: W0714 01:20:17.397065 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda005e203_ba4b_40b4_89ed_e2e661b38e42.slice/crio-bc69a814500ca7d0ed9070d7293287633e87a98c2d0d6beac1e28d9366a40e12 WatchSource:0}: Error finding container bc69a814500ca7d0ed9070d7293287633e87a98c2d0d6beac1e28d9366a40e12: Status 404 returned error can't find the container with id bc69a814500ca7d0ed9070d7293287633e87a98c2d0d6beac1e28d9366a40e12 Jul 14 01:20:17.399363 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.399330 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:20:17.399729 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.399411 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:20:17.399729 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.399462 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:20:17.445972 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:17.445943 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" event={"ID":"a005e203-ba4b-40b4-89ed-e2e661b38e42","Type":"ContainerStarted","Data":"bc69a814500ca7d0ed9070d7293287633e87a98c2d0d6beac1e28d9366a40e12"} Jul 14 01:20:18.452444 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:18.452407 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" event={"ID":"a005e203-ba4b-40b4-89ed-e2e661b38e42","Type":"ContainerStarted","Data":"10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77"} Jul 14 01:20:18.471232 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:18.471187 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" podStartSLOduration=2.471173252 podStartE2EDuration="2.471173252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:20:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:20:18.468968937 +0000 UTC m=+2155.320336299" watchObservedRunningTime="2026-07-14 01:20:18.471173252 +0000 UTC m=+2155.322540623" Jul 14 01:20:19.234223 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:19.234187 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:19.239417 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:19.239393 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:19.457783 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:19.457747 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:19.458743 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:19.458720 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:20.397661 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:20.397623 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:20.487970 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:20.487939 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:20.564054 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:20.564015 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:20.663674 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:20.663590 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:20.931094 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:20.931012 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:21.031507 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:21.031475 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:24.910604 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:24.910557 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:25.649118 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:25.649087 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:25.747689 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:25.747617 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:26.035332 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:26.035303 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:31.685116 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:31.685055 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:31.786478 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:31.786439 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:33.543243 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:33.542684 2557 generic.go:356] "Generic (PLEG): container finished" podID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerID="10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77" exitCode=0 Jul 14 01:20:33.543243 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:33.542857 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" event={"ID":"a005e203-ba4b-40b4-89ed-e2e661b38e42","Type":"ContainerDied","Data":"10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77"} Jul 14 01:20:33.547193 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:33.544265 2557 scope.go:117] "RemoveContainer" containerID="10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77" Jul 14 01:20:33.547193 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:33.545496 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:20:33.547193 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:33.545585 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:20:33.547193 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:33.545628 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:20:34.457903 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:34.457867 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:34.550329 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:34.550294 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" event={"ID":"a005e203-ba4b-40b4-89ed-e2e661b38e42","Type":"ContainerStarted","Data":"a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03"} Jul 14 01:20:35.234920 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:35.234886 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:35.235994 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:35.235967 2557 patch_prober.go:28] interesting pod/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 14 01:20:35.236125 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:35.236013 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" podUID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 14 01:20:36.235190 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:36.235130 2557 patch_prober.go:28] interesting pod/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 14 01:20:36.235593 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:36.235210 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" podUID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 14 01:20:37.235204 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:37.235144 2557 patch_prober.go:28] interesting pod/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body= Jul 14 01:20:37.235579 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:37.235230 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" podUID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" Jul 14 01:20:38.238617 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:38.238582 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:38.238993 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:38.238958 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:38.239575 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:38.239557 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:20:42.602551 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:42.602498 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:45.511457 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:45.511424 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:47.122520 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:47.122488 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:52.825511 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:52.825476 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:52.881744 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:52.881713 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:57.789791 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:57.789760 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r"] Jul 14 01:20:57.790309 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:57.790000 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" podUID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerName="istio-proxy" containerID="cri-o://a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03" gracePeriod=30 Jul 14 01:20:57.810413 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:57.810389 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:58.172174 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:58.172065 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:58.271558 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:58.271526 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:20:58.419080 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:20:58.419045 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:03.041113 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.041090 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:21:03.211401 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211334 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-token\") pod \"a005e203-ba4b-40b4-89ed-e2e661b38e42\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " Jul 14 01:21:03.211401 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211377 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-envoy\") pod \"a005e203-ba4b-40b4-89ed-e2e661b38e42\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " Jul 14 01:21:03.211401 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211399 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-podinfo\") pod \"a005e203-ba4b-40b4-89ed-e2e661b38e42\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " Jul 14 01:21:03.211659 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211417 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-socket\") pod \"a005e203-ba4b-40b4-89ed-e2e661b38e42\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " Jul 14 01:21:03.211659 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211431 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a005e203-ba4b-40b4-89ed-e2e661b38e42-istiod-ca-cert\") pod \"a005e203-ba4b-40b4-89ed-e2e661b38e42\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " Jul 14 01:21:03.211659 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211450 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-certs\") pod \"a005e203-ba4b-40b4-89ed-e2e661b38e42\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " Jul 14 01:21:03.211659 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211482 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-credential-socket\") pod \"a005e203-ba4b-40b4-89ed-e2e661b38e42\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " Jul 14 01:21:03.211659 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211510 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfd2w\" (UniqueName: \"kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-kube-api-access-sfd2w\") pod \"a005e203-ba4b-40b4-89ed-e2e661b38e42\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " Jul 14 01:21:03.211659 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211558 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-data\") pod \"a005e203-ba4b-40b4-89ed-e2e661b38e42\" (UID: \"a005e203-ba4b-40b4-89ed-e2e661b38e42\") " Jul 14 01:21:03.212026 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211732 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a005e203-ba4b-40b4-89ed-e2e661b38e42" (UID: "a005e203-ba4b-40b4-89ed-e2e661b38e42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:21:03.212026 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211791 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a005e203-ba4b-40b4-89ed-e2e661b38e42-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a005e203-ba4b-40b4-89ed-e2e661b38e42" (UID: "a005e203-ba4b-40b4-89ed-e2e661b38e42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:21:03.212026 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211852 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a005e203-ba4b-40b4-89ed-e2e661b38e42" (UID: "a005e203-ba4b-40b4-89ed-e2e661b38e42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:21:03.212026 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.211968 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a005e203-ba4b-40b4-89ed-e2e661b38e42" (UID: "a005e203-ba4b-40b4-89ed-e2e661b38e42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:21:03.212264 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.212039 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a005e203-ba4b-40b4-89ed-e2e661b38e42-istiod-ca-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:21:03.212264 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.212058 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-certs\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:21:03.212264 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.212072 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-credential-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:21:03.212598 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.212573 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-data" (OuterVolumeSpecName: "istio-data") pod "a005e203-ba4b-40b4-89ed-e2e661b38e42" (UID: "a005e203-ba4b-40b4-89ed-e2e661b38e42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:21:03.213560 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.213533 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-token" (OuterVolumeSpecName: "istio-token") pod "a005e203-ba4b-40b4-89ed-e2e661b38e42" (UID: "a005e203-ba4b-40b4-89ed-e2e661b38e42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:21:03.213560 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.213540 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a005e203-ba4b-40b4-89ed-e2e661b38e42" (UID: "a005e203-ba4b-40b4-89ed-e2e661b38e42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:21:03.213689 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.213571 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a005e203-ba4b-40b4-89ed-e2e661b38e42" (UID: "a005e203-ba4b-40b4-89ed-e2e661b38e42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:21:03.213818 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.213801 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-kube-api-access-sfd2w" (OuterVolumeSpecName: "kube-api-access-sfd2w") pod "a005e203-ba4b-40b4-89ed-e2e661b38e42" (UID: "a005e203-ba4b-40b4-89ed-e2e661b38e42"). InnerVolumeSpecName "kube-api-access-sfd2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:21:03.312837 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.312805 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:21:03.312837 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.312836 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-envoy\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:21:03.312837 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.312845 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-podinfo\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:21:03.313003 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.312854 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-workload-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:21:03.313003 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.312863 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfd2w\" (UniqueName: \"kubernetes.io/projected/a005e203-ba4b-40b4-89ed-e2e661b38e42-kube-api-access-sfd2w\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:21:03.313003 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.312872 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a005e203-ba4b-40b4-89ed-e2e661b38e42-istio-data\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:21:03.721367 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.721333 2557 generic.go:356] "Generic (PLEG): container finished" podID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerID="a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03" exitCode=0 Jul 14 01:21:03.721553 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.721390 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" event={"ID":"a005e203-ba4b-40b4-89ed-e2e661b38e42","Type":"ContainerDied","Data":"a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03"} Jul 14 01:21:03.721553 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.721459 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" event={"ID":"a005e203-ba4b-40b4-89ed-e2e661b38e42","Type":"ContainerDied","Data":"bc69a814500ca7d0ed9070d7293287633e87a98c2d0d6beac1e28d9366a40e12"} Jul 14 01:21:03.721553 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.721490 2557 scope.go:117] "RemoveContainer" containerID="a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03" Jul 14 01:21:03.721553 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.721411 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r" Jul 14 01:21:03.731332 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.731313 2557 scope.go:117] "RemoveContainer" containerID="10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77" Jul 14 01:21:03.736474 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.736452 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r"] Jul 14 01:21:03.739122 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.739104 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2a62f7c8-openshift-default-8677f8d979-lqd2r"] Jul 14 01:21:03.739984 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.739966 2557 scope.go:117] "RemoveContainer" containerID="a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03" Jul 14 01:21:03.740233 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:21:03.740212 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03\": container with ID starting with a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03 not found: ID does not exist" containerID="a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03" Jul 14 01:21:03.740309 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.740243 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03"} err="failed to get container status \"a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03\": rpc error: code = NotFound desc = could not find container \"a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03\": container with ID starting with a60a74188a36906e3a488adea0b61e7cf505c77872a5ebaad3479ee87804de03 not found: ID does not exist" Jul 14 01:21:03.740309 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.740260 2557 scope.go:117] "RemoveContainer" containerID="10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77" Jul 14 01:21:03.740494 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:21:03.740476 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77\": container with ID starting with 10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77 not found: ID does not exist" containerID="10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77" Jul 14 01:21:03.740538 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.740501 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77"} err="failed to get container status \"10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77\": rpc error: code = NotFound desc = could not find container \"10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77\": container with ID starting with 10035b0ec8a0ba31d8d7290b04d89efc5e04d43117f1c8e46ef073b3df448e77 not found: ID does not exist" Jul 14 01:21:03.955556 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:03.955481 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:04.056506 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:04.056459 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:04.155756 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:04.155725 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:04.314488 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:04.314456 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:05.678535 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:05.678504 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a005e203-ba4b-40b4-89ed-e2e661b38e42" path="/var/lib/kubelet/pods/a005e203-ba4b-40b4-89ed-e2e661b38e42/volumes" Jul 14 01:21:09.923851 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:09.923797 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:10.026638 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:10.026606 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:16.753175 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:16.753125 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:17.056139 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.056110 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh"] Jul 14 01:21:17.056577 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.056558 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerName="istio-proxy" Jul 14 01:21:17.056682 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.056579 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerName="istio-proxy" Jul 14 01:21:17.056743 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.056680 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerName="istio-proxy" Jul 14 01:21:17.060729 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.060711 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.062649 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.062627 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-48d1e8ae-openshift-default-dockercfg-7wt8j\"" Jul 14 01:21:17.073732 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.073707 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh"] Jul 14 01:21:17.084019 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.083995 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:17.225731 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.225692 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-credential-socket\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.225923 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.225743 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4598w\" (UniqueName: \"kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-kube-api-access-4598w\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.225923 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.225773 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-certs\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.225923 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.225797 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-envoy\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.225923 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.225820 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-podinfo\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.225923 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.225878 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istiod-ca-cert\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.226203 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.225944 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-socket\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.226203 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.225990 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-data\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.226203 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.226006 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-token\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.327393 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.327313 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-envoy\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.327393 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.327344 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-podinfo\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.327393 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.327370 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istiod-ca-cert\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.327671 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.327409 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-socket\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.327671 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.327448 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-data\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.327671 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.327583 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-token\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.327817 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.327703 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-credential-socket\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.327817 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.327753 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4598w\" (UniqueName: \"kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-kube-api-access-4598w\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.327817 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.327793 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-certs\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.327817 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.327803 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-data\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.328005 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.327821 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-socket\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.328166 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.328100 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-credential-socket\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.328520 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.328495 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-certs\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.328691 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.328667 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istiod-ca-cert\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.330170 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.330132 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-podinfo\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.330298 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.330174 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-envoy\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.334769 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.334746 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-token\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.334992 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.334971 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4598w\" (UniqueName: \"kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-kube-api-access-4598w\") pod \"e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.372328 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.372300 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:17.374514 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.374486 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:17.522744 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.522719 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh"] Jul 14 01:21:17.524378 ip-10-0-134-143 kubenswrapper[2557]: W0714 01:21:17.524351 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67bcd6ff_cb0c_4ba6_b353_7581626b22a7.slice/crio-f2b9ef805d92cdaa0e21b77e22436e622701c2f67e8e07d643722033ece89cc3 WatchSource:0}: Error finding container f2b9ef805d92cdaa0e21b77e22436e622701c2f67e8e07d643722033ece89cc3: Status 404 returned error can't find the container with id f2b9ef805d92cdaa0e21b77e22436e622701c2f67e8e07d643722033ece89cc3 Jul 14 01:21:17.526583 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.526547 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:21:17.526668 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.526618 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:21:17.526668 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.526650 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:21:17.806554 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.806519 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" event={"ID":"67bcd6ff-cb0c-4ba6-b353-7581626b22a7","Type":"ContainerStarted","Data":"69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7"} Jul 14 01:21:17.806926 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.806571 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" event={"ID":"67bcd6ff-cb0c-4ba6-b353-7581626b22a7","Type":"ContainerStarted","Data":"f2b9ef805d92cdaa0e21b77e22436e622701c2f67e8e07d643722033ece89cc3"} Jul 14 01:21:17.824952 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:17.824887 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" podStartSLOduration=0.824867287 podStartE2EDuration="824.867287ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:21:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:21:17.821990504 +0000 UTC m=+2214.673357873" watchObservedRunningTime="2026-07-14 01:21:17.824867287 +0000 UTC m=+2214.676234659" Jul 14 01:21:18.372816 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:18.372782 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:18.377427 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:18.377405 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:18.812396 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:18.812368 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:18.813245 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:18.813226 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:19.748320 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:19.748283 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:20.228473 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:20.228436 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:20.333860 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:20.333832 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:20.493588 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:20.493506 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:20.595467 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:20.595436 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:21.467351 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:21.467317 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:21.558652 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:21.558620 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:22.319824 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:22.319793 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:22.417690 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:22.417653 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:24.289652 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:24.289620 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:27.918994 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:27.918955 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:30.890218 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:30.890188 2557 generic.go:356] "Generic (PLEG): container finished" podID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerID="69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7" exitCode=0 Jul 14 01:21:30.890714 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:30.890270 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" event={"ID":"67bcd6ff-cb0c-4ba6-b353-7581626b22a7","Type":"ContainerDied","Data":"69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7"} Jul 14 01:21:30.890806 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:30.890754 2557 scope.go:117] "RemoveContainer" containerID="69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7" Jul 14 01:21:30.891702 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:30.891679 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:21:30.891758 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:30.891743 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:21:30.891794 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:30.891773 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:21:31.901419 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:31.901388 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" event={"ID":"67bcd6ff-cb0c-4ba6-b353-7581626b22a7","Type":"ContainerStarted","Data":"c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116"} Jul 14 01:21:32.373932 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:32.373891 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:32.374132 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:32.374014 2557 patch_prober.go:28] interesting pod/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 14 01:21:32.374132 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:32.374055 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 14 01:21:33.373547 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:33.373510 2557 patch_prober.go:28] interesting pod/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 14 01:21:33.374117 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:33.374083 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 14 01:21:33.813248 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:33.813219 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:34.372990 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:34.372953 2557 patch_prober.go:28] interesting pod/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 14 01:21:34.373327 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:34.373293 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 14 01:21:35.373601 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:35.373561 2557 patch_prober.go:28] interesting pod/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body= Jul 14 01:21:35.374105 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:35.373631 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" Jul 14 01:21:36.376785 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:36.376756 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:36.377950 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:36.377930 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:21:43.818778 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:43.818741 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:21:43.867590 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:21:43.867563 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:22:03.105247 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:22:03.105172 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:22:03.158300 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:22:03.158269 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:22:23.459585 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:22:23.459550 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:22:27.960793 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:22:27.960753 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:22:33.039098 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:22:33.039062 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:09.376735 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:09.376698 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:09.721498 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:09.721401 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:15.248965 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:15.248874 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:15.542373 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:15.542338 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:15.609505 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:15.609474 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:20.915575 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:20.915538 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:20.974466 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:20.974428 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:30.201996 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:30.201960 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh"] Jul 14 01:23:30.202501 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:30.202226 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerName="istio-proxy" containerID="cri-o://c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116" gracePeriod=30 Jul 14 01:23:30.228835 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:30.228803 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:31.737240 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:31.737199 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:32.395435 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:32.395399 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:32.494248 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:32.494210 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:32.595953 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:32.595907 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:33.992254 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:33.992218 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:35.465283 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.465255 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:23:35.550221 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550185 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istiod-ca-cert\") pod \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " Jul 14 01:23:35.550399 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550234 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4598w\" (UniqueName: \"kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-kube-api-access-4598w\") pod \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " Jul 14 01:23:35.550399 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550293 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-podinfo\") pod \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " Jul 14 01:23:35.550399 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550363 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-credential-socket\") pod \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " Jul 14 01:23:35.550399 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550397 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-envoy\") pod \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " Jul 14 01:23:35.550632 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550447 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-socket\") pod \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " Jul 14 01:23:35.550632 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550486 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-token\") pod \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " Jul 14 01:23:35.550632 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550545 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-certs\") pod \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " Jul 14 01:23:35.550632 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550546 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67bcd6ff-cb0c-4ba6-b353-7581626b22a7" (UID: "67bcd6ff-cb0c-4ba6-b353-7581626b22a7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:23:35.550632 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550582 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-data\") pod \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\" (UID: \"67bcd6ff-cb0c-4ba6-b353-7581626b22a7\") " Jul 14 01:23:35.550895 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550647 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67bcd6ff-cb0c-4ba6-b353-7581626b22a7" (UID: "67bcd6ff-cb0c-4ba6-b353-7581626b22a7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:23:35.550950 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550916 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istiod-ca-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:23:35.550950 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550941 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-credential-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:23:35.551055 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.550971 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67bcd6ff-cb0c-4ba6-b353-7581626b22a7" (UID: "67bcd6ff-cb0c-4ba6-b353-7581626b22a7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:23:35.551131 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.551103 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67bcd6ff-cb0c-4ba6-b353-7581626b22a7" (UID: "67bcd6ff-cb0c-4ba6-b353-7581626b22a7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:23:35.551701 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.551674 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-data" (OuterVolumeSpecName: "istio-data") pod "67bcd6ff-cb0c-4ba6-b353-7581626b22a7" (UID: "67bcd6ff-cb0c-4ba6-b353-7581626b22a7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:23:35.552578 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.552548 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-kube-api-access-4598w" (OuterVolumeSpecName: "kube-api-access-4598w") pod "67bcd6ff-cb0c-4ba6-b353-7581626b22a7" (UID: "67bcd6ff-cb0c-4ba6-b353-7581626b22a7"). InnerVolumeSpecName "kube-api-access-4598w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:23:35.552920 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.552897 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67bcd6ff-cb0c-4ba6-b353-7581626b22a7" (UID: "67bcd6ff-cb0c-4ba6-b353-7581626b22a7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:23:35.553001 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.552961 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67bcd6ff-cb0c-4ba6-b353-7581626b22a7" (UID: "67bcd6ff-cb0c-4ba6-b353-7581626b22a7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:23:35.553045 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.553034 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-token" (OuterVolumeSpecName: "istio-token") pod "67bcd6ff-cb0c-4ba6-b353-7581626b22a7" (UID: "67bcd6ff-cb0c-4ba6-b353-7581626b22a7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:23:35.569582 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.569553 2557 generic.go:356] "Generic (PLEG): container finished" podID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerID="c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116" exitCode=0 Jul 14 01:23:35.569697 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.569605 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" Jul 14 01:23:35.569697 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.569648 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" event={"ID":"67bcd6ff-cb0c-4ba6-b353-7581626b22a7","Type":"ContainerDied","Data":"c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116"} Jul 14 01:23:35.569811 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.569748 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh" event={"ID":"67bcd6ff-cb0c-4ba6-b353-7581626b22a7","Type":"ContainerDied","Data":"f2b9ef805d92cdaa0e21b77e22436e622701c2f67e8e07d643722033ece89cc3"} Jul 14 01:23:35.569811 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.569783 2557 scope.go:117] "RemoveContainer" containerID="c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116" Jul 14 01:23:35.580301 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.580281 2557 scope.go:117] "RemoveContainer" containerID="69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7" Jul 14 01:23:35.590652 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.590633 2557 scope.go:117] "RemoveContainer" containerID="c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116" Jul 14 01:23:35.591270 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:23:35.591117 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116\": container with ID starting with c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116 not found: ID does not exist" containerID="c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116" Jul 14 01:23:35.591270 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.591172 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116"} err="failed to get container status \"c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116\": rpc error: code = NotFound desc = could not find container \"c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116\": container with ID starting with c842065057838276d7f5e0ef6520ee5e311c39e2b921ffe5438a3580db310116 not found: ID does not exist" Jul 14 01:23:35.591270 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.591197 2557 scope.go:117] "RemoveContainer" containerID="69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7" Jul 14 01:23:35.591513 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:23:35.591490 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7\": container with ID starting with 69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7 not found: ID does not exist" containerID="69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7" Jul 14 01:23:35.591564 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.591524 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7"} err="failed to get container status \"69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7\": rpc error: code = NotFound desc = could not find container \"69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7\": container with ID starting with 69269671007d92bab45b17997f18c1a7a3d0e79d4d87b17832dbf8bfb5628ec7 not found: ID does not exist" Jul 14 01:23:35.593478 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.593458 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh"] Jul 14 01:23:35.597004 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.596985 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-48d1e8ae-openshift-default-559c47694c-fbnlh"] Jul 14 01:23:35.651485 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.651456 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-certs\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:23:35.651485 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.651487 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-data\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:23:35.651673 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.651500 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4598w\" (UniqueName: \"kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-kube-api-access-4598w\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:23:35.651673 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.651513 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-podinfo\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:23:35.651673 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.651525 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-envoy\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:23:35.651673 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.651536 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-workload-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:23:35.651673 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.651550 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67bcd6ff-cb0c-4ba6-b353-7581626b22a7-istio-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:23:35.678903 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:35.678875 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" path="/var/lib/kubelet/pods/67bcd6ff-cb0c-4ba6-b353-7581626b22a7/volumes" Jul 14 01:23:39.674203 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:39.674164 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:39.767406 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:39.767378 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:40.367322 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:40.367287 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:46.323121 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:46.323083 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:46.427766 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:46.427734 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:46.523939 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:46.523906 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:57.036432 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:57.036393 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:23:57.396078 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:23:57.395995 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:18.383625 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:18.383580 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:23.777442 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:23.777418 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:24:23.787083 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:23.787059 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:24:24.180842 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.180805 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:24.242617 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.242584 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:24.543765 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.543721 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:24.789216 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.789186 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp"] Jul 14 01:24:24.789775 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.789752 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerName="istio-proxy" Jul 14 01:24:24.789775 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.789770 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerName="istio-proxy" Jul 14 01:24:24.789955 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.789785 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerName="istio-proxy" Jul 14 01:24:24.789955 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.789794 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerName="istio-proxy" Jul 14 01:24:24.789955 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.789821 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerName="istio-proxy" Jul 14 01:24:24.789955 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.789830 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerName="istio-proxy" Jul 14 01:24:24.789955 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.789924 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerName="istio-proxy" Jul 14 01:24:24.789955 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.789936 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="67bcd6ff-cb0c-4ba6-b353-7581626b22a7" containerName="istio-proxy" Jul 14 01:24:24.789955 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.789950 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a005e203-ba4b-40b4-89ed-e2e661b38e42" containerName="istio-proxy" Jul 14 01:24:24.792925 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.792906 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:24.794657 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.794601 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-da3fe6-openshift-default-dockercfg-vtmkh\"" Jul 14 01:24:24.806357 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.806330 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp"] Jul 14 01:24:24.812657 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.812633 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:24.951033 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.951002 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-data\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:24.951033 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.951033 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-podinfo\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:24.951253 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.951054 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istiod-ca-cert\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:24.951253 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.951134 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-token\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:24.951253 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.951193 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-credential-socket\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:24.951253 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.951228 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-certs\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:24.951389 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.951259 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-envoy\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:24.951389 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.951291 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kjzd\" (UniqueName: \"kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-kube-api-access-6kjzd\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:24.951389 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.951375 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-socket\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:24.959979 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:24.959950 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:25.052266 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.052200 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-certs\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.052266 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.052237 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-envoy\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.052266 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.052257 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kjzd\" (UniqueName: \"kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-kube-api-access-6kjzd\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.052477 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.052304 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-socket\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.052477 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.052331 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-data\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.052477 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.052348 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-podinfo\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.052477 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.052374 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istiod-ca-cert\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.052477 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.052424 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-token\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.052477 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.052476 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-credential-socket\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.052780 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.052659 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-certs\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.052780 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.052743 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-socket\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.053032 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.053011 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-data\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.053227 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.053207 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istiod-ca-cert\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.053282 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.053268 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-credential-socket\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.054457 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.054441 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-envoy\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.054790 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.054768 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-podinfo\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.071639 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.071618 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-token\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.072039 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.072016 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kjzd\" (UniqueName: \"kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-kube-api-access-6kjzd\") pod \"e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.114722 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.114696 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:25.249954 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.249927 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp"] Jul 14 01:24:25.251287 ip-10-0-134-143 kubenswrapper[2557]: W0714 01:24:25.251258 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5151bf2f_5e12_4bbf_8041_bb1dfd77346d.slice/crio-7541237f9bf5721fe29e3d040da37054264e0ed55ded3b53964a56c7b256d1d2 WatchSource:0}: Error finding container 7541237f9bf5721fe29e3d040da37054264e0ed55ded3b53964a56c7b256d1d2: Status 404 returned error can't find the container with id 7541237f9bf5721fe29e3d040da37054264e0ed55ded3b53964a56c7b256d1d2 Jul 14 01:24:25.253472 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.253442 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:24:25.253556 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.253504 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:24:25.253556 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.253533 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:24:25.834916 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.834885 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" event={"ID":"5151bf2f-5e12-4bbf-8041-bb1dfd77346d","Type":"ContainerStarted","Data":"26e0c678b985322d041bd6114499f22fa0b70f1ebd215901080d9ae573cc5158"} Jul 14 01:24:25.835302 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.834932 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" event={"ID":"5151bf2f-5e12-4bbf-8041-bb1dfd77346d","Type":"ContainerStarted","Data":"7541237f9bf5721fe29e3d040da37054264e0ed55ded3b53964a56c7b256d1d2"} Jul 14 01:24:25.851281 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:25.851230 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" podStartSLOduration=1.851211863 podStartE2EDuration="1.851211863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:24:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:24:25.850693731 +0000 UTC m=+2402.702061101" watchObservedRunningTime="2026-07-14 01:24:25.851211863 +0000 UTC m=+2402.702579233" Jul 14 01:24:26.115169 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:26.115072 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:26.119840 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:26.119816 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:26.840183 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:26.840138 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:26.841144 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:26.841124 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:27.850652 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:27.850619 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:27.926542 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:27.926508 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:28.002661 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:28.002616 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:28.372788 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:28.372754 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:28.487949 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:28.487891 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:28.588635 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:28.588603 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:32.269230 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:32.269193 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:32.601641 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:32.601603 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:32.962060 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:32.961978 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:33.073843 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:33.073783 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:36.104801 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:36.104765 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:36.187797 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:36.187763 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:36.260374 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:36.260341 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:36.352844 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:36.352811 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:36.745597 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:36.745543 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:36.842464 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:36.842426 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:40.625999 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:40.625958 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:42.881331 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:42.881290 2557 patch_prober.go:28] interesting pod/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 01:24:42.881793 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:42.881374 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 01:24:43.940139 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:43.940084 2557 generic.go:356] "Generic (PLEG): container finished" podID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerID="26e0c678b985322d041bd6114499f22fa0b70f1ebd215901080d9ae573cc5158" exitCode=0 Jul 14 01:24:43.940500 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:43.940167 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" event={"ID":"5151bf2f-5e12-4bbf-8041-bb1dfd77346d","Type":"ContainerDied","Data":"26e0c678b985322d041bd6114499f22fa0b70f1ebd215901080d9ae573cc5158"} Jul 14 01:24:43.940545 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:43.940506 2557 scope.go:117] "RemoveContainer" containerID="26e0c678b985322d041bd6114499f22fa0b70f1ebd215901080d9ae573cc5158" Jul 14 01:24:43.941424 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:43.941400 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:24:43.941521 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:43.941468 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:24:43.941521 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:43.941516 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:24:44.947905 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:44.947871 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" event={"ID":"5151bf2f-5e12-4bbf-8041-bb1dfd77346d","Type":"ContainerStarted","Data":"dbf121b4e5df9d33960b5b06eb98ca74729551a12429d05fd84e29eb937fcda8"} Jul 14 01:24:45.115304 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:45.115262 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:45.115545 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:45.115522 2557 patch_prober.go:28] interesting pod/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 14 01:24:45.115634 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:45.115576 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 14 01:24:46.115835 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:46.115802 2557 patch_prober.go:28] interesting pod/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 14 01:24:46.116241 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:46.115856 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 14 01:24:47.115749 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:47.115719 2557 patch_prober.go:28] interesting pod/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 14 01:24:47.115952 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:47.115769 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 14 01:24:48.116105 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:48.116072 2557 patch_prober.go:28] interesting pod/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body= Jul 14 01:24:48.116506 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:48.116125 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" Jul 14 01:24:49.118829 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:49.118800 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:49.119319 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:49.119095 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:49.119767 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:49.119747 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:24:49.439833 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:49.439741 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:49.534281 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:49.534252 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:57.425378 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:57.425339 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:24:57.526692 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:24:57.526653 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:25:02.760048 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:25:02.760007 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:25:34.833053 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:25:34.833019 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:25:40.331164 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:25:40.331105 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:25:40.430945 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:25:40.430914 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:25:54.376897 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:25:54.376840 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:25:55.420344 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:25:55.420304 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:25:56.099685 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:25:56.099644 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:25:56.197814 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:25:56.197780 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:05.943349 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:05.943310 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:10.626809 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:10.626779 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:16.464385 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:16.464350 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:16.517312 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:16.517278 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:21.296138 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:21.296105 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp"] Jul 14 01:26:21.296532 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:21.296361 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" containerID="cri-o://dbf121b4e5df9d33960b5b06eb98ca74729551a12429d05fd84e29eb937fcda8" gracePeriod=30 Jul 14 01:26:21.312208 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:21.312145 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:21.537594 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:21.537558 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:22.419530 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:22.419497 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:22.517528 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:22.517495 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:23.136723 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:23.136686 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:23.238040 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:23.238002 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:23.981552 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:23.981520 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:24.082662 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:24.082635 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:26.503062 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.503031 2557 generic.go:356] "Generic (PLEG): container finished" podID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerID="dbf121b4e5df9d33960b5b06eb98ca74729551a12429d05fd84e29eb937fcda8" exitCode=0 Jul 14 01:26:26.503511 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.503099 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" event={"ID":"5151bf2f-5e12-4bbf-8041-bb1dfd77346d","Type":"ContainerDied","Data":"dbf121b4e5df9d33960b5b06eb98ca74729551a12429d05fd84e29eb937fcda8"} Jul 14 01:26:26.503511 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.503213 2557 scope.go:117] "RemoveContainer" containerID="26e0c678b985322d041bd6114499f22fa0b70f1ebd215901080d9ae573cc5158" Jul 14 01:26:26.560277 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.560224 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:26:26.646056 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646029 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-certs\") pod \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " Jul 14 01:26:26.646217 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646073 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-data\") pod \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " Jul 14 01:26:26.646217 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646107 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-token\") pod \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " Jul 14 01:26:26.646217 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646174 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istiod-ca-cert\") pod \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " Jul 14 01:26:26.646217 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646211 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-envoy\") pod \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " Jul 14 01:26:26.646441 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646247 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-credential-socket\") pod \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " Jul 14 01:26:26.646441 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646285 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-podinfo\") pod \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " Jul 14 01:26:26.646441 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646314 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-socket\") pod \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " Jul 14 01:26:26.646441 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646342 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5151bf2f-5e12-4bbf-8041-bb1dfd77346d" (UID: "5151bf2f-5e12-4bbf-8041-bb1dfd77346d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:26:26.646441 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646397 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kjzd\" (UniqueName: \"kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-kube-api-access-6kjzd\") pod \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\" (UID: \"5151bf2f-5e12-4bbf-8041-bb1dfd77346d\") " Jul 14 01:26:26.646717 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646619 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5151bf2f-5e12-4bbf-8041-bb1dfd77346d" (UID: "5151bf2f-5e12-4bbf-8041-bb1dfd77346d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:26:26.646717 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646668 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5151bf2f-5e12-4bbf-8041-bb1dfd77346d" (UID: "5151bf2f-5e12-4bbf-8041-bb1dfd77346d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:26:26.646875 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646726 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5151bf2f-5e12-4bbf-8041-bb1dfd77346d" (UID: "5151bf2f-5e12-4bbf-8041-bb1dfd77346d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:26:26.646875 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646760 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-certs\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:26:26.646875 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646782 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istiod-ca-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:26:26.646875 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646795 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-workload-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:26:26.647083 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.646912 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-data" (OuterVolumeSpecName: "istio-data") pod "5151bf2f-5e12-4bbf-8041-bb1dfd77346d" (UID: "5151bf2f-5e12-4bbf-8041-bb1dfd77346d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:26:26.648387 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.648363 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-token" (OuterVolumeSpecName: "istio-token") pod "5151bf2f-5e12-4bbf-8041-bb1dfd77346d" (UID: "5151bf2f-5e12-4bbf-8041-bb1dfd77346d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:26:26.648493 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.648424 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5151bf2f-5e12-4bbf-8041-bb1dfd77346d" (UID: "5151bf2f-5e12-4bbf-8041-bb1dfd77346d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:26:26.648493 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.648442 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5151bf2f-5e12-4bbf-8041-bb1dfd77346d" (UID: "5151bf2f-5e12-4bbf-8041-bb1dfd77346d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:26:26.648610 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.648592 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-kube-api-access-6kjzd" (OuterVolumeSpecName: "kube-api-access-6kjzd") pod "5151bf2f-5e12-4bbf-8041-bb1dfd77346d" (UID: "5151bf2f-5e12-4bbf-8041-bb1dfd77346d"). InnerVolumeSpecName "kube-api-access-6kjzd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:26:26.747978 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.747955 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-data\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:26:26.747978 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.747976 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:26:26.748107 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.747984 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-envoy\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:26:26.748107 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.747993 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-credential-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:26:26.748107 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.748001 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-istio-podinfo\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:26:26.748107 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:26.748010 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kjzd\" (UniqueName: \"kubernetes.io/projected/5151bf2f-5e12-4bbf-8041-bb1dfd77346d-kube-api-access-6kjzd\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:26:27.510028 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:27.510003 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" Jul 14 01:26:27.510440 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:27.510049 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp" event={"ID":"5151bf2f-5e12-4bbf-8041-bb1dfd77346d","Type":"ContainerDied","Data":"7541237f9bf5721fe29e3d040da37054264e0ed55ded3b53964a56c7b256d1d2"} Jul 14 01:26:27.510440 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:27.510104 2557 scope.go:117] "RemoveContainer" containerID="dbf121b4e5df9d33960b5b06eb98ca74729551a12429d05fd84e29eb937fcda8" Jul 14 01:26:27.531476 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:27.531451 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp"] Jul 14 01:26:27.534511 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:27.534489 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-da3fe6-openshift-default-55bfb7d6d7-hs7sp"] Jul 14 01:26:27.678185 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:27.678141 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" path="/var/lib/kubelet/pods/5151bf2f-5e12-4bbf-8041-bb1dfd77346d/volumes" Jul 14 01:26:32.077075 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:32.077037 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:36.843583 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:36.843549 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:42.419177 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:42.419122 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:42.475001 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:42.474969 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:47.453295 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:47.453245 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:47.677190 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:47.677135 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:48.004337 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.004300 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:48.276136 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.276042 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q"] Jul 14 01:26:48.276642 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.276617 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" Jul 14 01:26:48.276815 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.276800 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" Jul 14 01:26:48.276910 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.276877 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" Jul 14 01:26:48.276910 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.276892 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" Jul 14 01:26:48.277046 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.277030 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" Jul 14 01:26:48.279488 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.279467 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.282623 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.282600 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-cb7605-openshift-default-dockercfg-zd9xf\"" Jul 14 01:26:48.290305 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.290281 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q"] Jul 14 01:26:48.297229 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.297203 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:48.422680 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.422645 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-socket\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.422861 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.422693 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-credential-socket\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.422861 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.422778 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-envoy\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.422861 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.422827 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80d18117-5960-4b16-b4ac-426a94c0020a-istio-podinfo\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.423000 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.422888 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80d18117-5960-4b16-b4ac-426a94c0020a-istiod-ca-cert\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.423000 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.422960 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-data\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.423000 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.422984 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2zz6\" (UniqueName: \"kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-kube-api-access-j2zz6\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.423142 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.423020 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-certs\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.423142 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.423041 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-istio-token\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.423460 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.423434 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:48.523881 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.523856 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2zz6\" (UniqueName: \"kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-kube-api-access-j2zz6\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524308 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.523892 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-certs\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524308 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.523911 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-istio-token\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524308 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.523954 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-socket\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524308 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.523981 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-credential-socket\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524308 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.524029 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-envoy\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524308 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.524072 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80d18117-5960-4b16-b4ac-426a94c0020a-istio-podinfo\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524308 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.524108 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80d18117-5960-4b16-b4ac-426a94c0020a-istiod-ca-cert\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524308 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.524208 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-data\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524720 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.524326 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-certs\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524720 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.524410 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-credential-socket\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524720 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.524431 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-socket\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524720 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.524553 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-data\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.524884 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.524867 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80d18117-5960-4b16-b4ac-426a94c0020a-istiod-ca-cert\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.526327 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.526268 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-envoy\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.526521 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.526502 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80d18117-5960-4b16-b4ac-426a94c0020a-istio-podinfo\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.530400 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.530380 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-istio-token\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.530533 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.530517 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2zz6\" (UniqueName: \"kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-kube-api-access-j2zz6\") pod \"e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.590976 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.590947 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:48.725833 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.725810 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q"] Jul 14 01:26:48.727402 ip-10-0-134-143 kubenswrapper[2557]: W0714 01:26:48.727371 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80d18117_5960_4b16_b4ac_426a94c0020a.slice/crio-ae4699220b7a721d1443a9c825d8a8cef8c90f67c2cd64d55529eff8c6dca0da WatchSource:0}: Error finding container ae4699220b7a721d1443a9c825d8a8cef8c90f67c2cd64d55529eff8c6dca0da: Status 404 returned error can't find the container with id ae4699220b7a721d1443a9c825d8a8cef8c90f67c2cd64d55529eff8c6dca0da Jul 14 01:26:48.729432 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.729401 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:26:48.729520 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.729462 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:26:48.729520 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:48.729492 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:26:49.644476 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:49.644434 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" event={"ID":"80d18117-5960-4b16-b4ac-426a94c0020a","Type":"ContainerStarted","Data":"64ea195b96dc1c5fa010eb3b2577566413c94bc9c604e0e078eb0d0cb4d0e899"} Jul 14 01:26:49.644938 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:49.644496 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" event={"ID":"80d18117-5960-4b16-b4ac-426a94c0020a","Type":"ContainerStarted","Data":"ae4699220b7a721d1443a9c825d8a8cef8c90f67c2cd64d55529eff8c6dca0da"} Jul 14 01:26:49.663749 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:49.663702 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" podStartSLOduration=1.663686606 podStartE2EDuration="1.663686606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:26:49.66134467 +0000 UTC m=+2546.512712040" watchObservedRunningTime="2026-07-14 01:26:49.663686606 +0000 UTC m=+2546.515053975" Jul 14 01:26:50.591903 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:50.591866 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:50.596829 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:50.596801 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:50.650214 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:50.650189 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:50.650829 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:50.650811 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:26:51.583921 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:51.583873 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:51.679051 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:51.679023 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:51.786860 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:51.786823 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:51.895037 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:51.894958 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:52.335570 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:52.335537 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:52.435988 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:52.435961 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:56.073382 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.073325 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:56.529049 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.528959 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:56.787338 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.787305 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn"] Jul 14 01:26:56.788016 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.787993 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5151bf2f-5e12-4bbf-8041-bb1dfd77346d" containerName="istio-proxy" Jul 14 01:26:56.790651 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.790631 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.792327 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.792308 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-e7c477-openshift-default-dockercfg-bs8rh\"" Jul 14 01:26:56.801807 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.801784 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn"] Jul 14 01:26:56.816786 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.816758 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:56.896901 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.896867 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-data\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.897077 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.896929 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-certs\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.897077 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.897016 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-credential-socket\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.897077 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.897049 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd537788-34cd-42f5-af19-92bae1b3f98b-istiod-ca-cert\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.897277 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.897094 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-socket\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.897277 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.897133 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-token\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.897277 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.897189 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27m8t\" (UniqueName: \"kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-kube-api-access-27m8t\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.897277 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.897243 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-envoy\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.897485 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.897285 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-podinfo\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.998437 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.998406 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-envoy\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.998583 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.998448 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-podinfo\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.998583 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.998479 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-data\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.998583 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.998510 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-certs\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.998583 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.998537 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-credential-socket\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.998583 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.998573 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd537788-34cd-42f5-af19-92bae1b3f98b-istiod-ca-cert\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.998800 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.998618 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-socket\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.998800 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.998645 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-token\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.998800 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.998673 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27m8t\" (UniqueName: \"kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-kube-api-access-27m8t\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.999127 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.999103 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-certs\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.999327 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.999172 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-data\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.999434 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.999367 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-credential-socket\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.999511 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.999492 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-socket\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:56.999551 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:56.999533 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd537788-34cd-42f5-af19-92bae1b3f98b-istiod-ca-cert\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:57.001046 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.001026 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-podinfo\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:57.001135 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.001112 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-envoy\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:57.005236 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.005210 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-token\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:57.005516 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.005496 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27m8t\" (UniqueName: \"kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-kube-api-access-27m8t\") pod \"e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:57.094081 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.094017 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:57.103041 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.103023 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:57.247980 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.247956 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn"] Jul 14 01:26:57.250163 ip-10-0-134-143 kubenswrapper[2557]: W0714 01:26:57.250122 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd537788_34cd_42f5_af19_92bae1b3f98b.slice/crio-dd0df0d181ef3f38a7c9fb9db410a557255b3139c886f4494d125bf17eb5af47 WatchSource:0}: Error finding container dd0df0d181ef3f38a7c9fb9db410a557255b3139c886f4494d125bf17eb5af47: Status 404 returned error can't find the container with id dd0df0d181ef3f38a7c9fb9db410a557255b3139c886f4494d125bf17eb5af47 Jul 14 01:26:57.252676 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.252645 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:26:57.252747 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.252710 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:26:57.252747 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.252737 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:26:57.688996 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.688959 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" event={"ID":"fd537788-34cd-42f5-af19-92bae1b3f98b","Type":"ContainerStarted","Data":"cef10afc5b1ad74bc40f6a5d1b006b6c4d7fcfd595e6eaf25ace40388ba73f53"} Jul 14 01:26:57.689189 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.689019 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" event={"ID":"fd537788-34cd-42f5-af19-92bae1b3f98b","Type":"ContainerStarted","Data":"dd0df0d181ef3f38a7c9fb9db410a557255b3139c886f4494d125bf17eb5af47"} Jul 14 01:26:57.708342 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:57.708291 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" podStartSLOduration=1.708270627 podStartE2EDuration="1.708270627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:26:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:26:57.705444146 +0000 UTC m=+2554.556811515" watchObservedRunningTime="2026-07-14 01:26:57.708270627 +0000 UTC m=+2554.559637998" Jul 14 01:26:58.103333 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:58.103301 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:58.108485 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:58.108455 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:58.696785 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:58.696738 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:58.697666 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:58.697643 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:26:59.607856 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:59.607822 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:59.712391 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:59.712360 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:59.809944 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:59.809906 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:26:59.915516 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:26:59.915417 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:00.232590 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:00.232492 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:00.333696 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:00.333665 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:04.105540 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:04.105503 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:05.651813 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:05.651780 2557 patch_prober.go:28] interesting pod/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 14 01:27:05.652289 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:05.651832 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 14 01:27:05.745833 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:05.745798 2557 generic.go:356] "Generic (PLEG): container finished" podID="80d18117-5960-4b16-b4ac-426a94c0020a" containerID="64ea195b96dc1c5fa010eb3b2577566413c94bc9c604e0e078eb0d0cb4d0e899" exitCode=0 Jul 14 01:27:05.745990 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:05.745872 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" event={"ID":"80d18117-5960-4b16-b4ac-426a94c0020a","Type":"ContainerDied","Data":"64ea195b96dc1c5fa010eb3b2577566413c94bc9c604e0e078eb0d0cb4d0e899"} Jul 14 01:27:05.746317 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:05.746303 2557 scope.go:117] "RemoveContainer" containerID="64ea195b96dc1c5fa010eb3b2577566413c94bc9c604e0e078eb0d0cb4d0e899" Jul 14 01:27:05.747323 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:05.747296 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:27:05.747438 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:05.747361 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:27:05.747438 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:05.747412 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:27:06.758313 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:06.758266 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" event={"ID":"80d18117-5960-4b16-b4ac-426a94c0020a","Type":"ContainerStarted","Data":"62f3f1bca1753e05bdae900620657a515cc0e108a246dbb267e7b7f2dfbdae80"} Jul 14 01:27:07.591901 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:07.591864 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:27:07.592357 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:07.592328 2557 patch_prober.go:28] interesting pod/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 14 01:27:07.592480 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:07.592415 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 14 01:27:08.591544 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:08.591503 2557 patch_prober.go:28] interesting pod/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 14 01:27:08.592523 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:08.592122 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 14 01:27:09.591966 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:09.591933 2557 patch_prober.go:28] interesting pod/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 14 01:27:09.592543 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:09.592511 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 14 01:27:10.592363 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:10.592325 2557 patch_prober.go:28] interesting pod/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body= Jul 14 01:27:10.593368 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:10.592922 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" Jul 14 01:27:11.595797 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:11.595767 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:27:11.596280 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:11.596266 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:27:11.596945 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:11.596927 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:27:12.565733 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:12.565707 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:12.668617 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:12.668544 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:12.798608 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:12.798579 2557 generic.go:356] "Generic (PLEG): container finished" podID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerID="cef10afc5b1ad74bc40f6a5d1b006b6c4d7fcfd595e6eaf25ace40388ba73f53" exitCode=0 Jul 14 01:27:12.798781 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:12.798663 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" event={"ID":"fd537788-34cd-42f5-af19-92bae1b3f98b","Type":"ContainerDied","Data":"cef10afc5b1ad74bc40f6a5d1b006b6c4d7fcfd595e6eaf25ace40388ba73f53"} Jul 14 01:27:12.799122 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:12.799105 2557 scope.go:117] "RemoveContainer" containerID="cef10afc5b1ad74bc40f6a5d1b006b6c4d7fcfd595e6eaf25ace40388ba73f53" Jul 14 01:27:12.800231 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:12.800193 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:27:12.800313 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:12.800279 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:27:12.800355 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:12.800327 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:27:13.696978 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:13.696938 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:27:13.806537 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:13.806498 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" event={"ID":"fd537788-34cd-42f5-af19-92bae1b3f98b","Type":"ContainerStarted","Data":"a46779299f3112b9347436389592dca12faf68f9639d5598ae36634498e9fda6"} Jul 14 01:27:14.103894 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:14.103853 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:27:14.105040 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:14.105017 2557 patch_prober.go:28] interesting pod/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 14 01:27:14.105181 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:14.105059 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 14 01:27:15.103963 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:15.103928 2557 patch_prober.go:28] interesting pod/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 14 01:27:15.104359 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:15.103981 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 14 01:27:16.103915 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:16.103878 2557 patch_prober.go:28] interesting pod/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 14 01:27:16.104086 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:16.103944 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 14 01:27:17.103373 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:17.103338 2557 patch_prober.go:28] interesting pod/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body= Jul 14 01:27:17.103552 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:17.103399 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" Jul 14 01:27:18.107626 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:18.107600 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:27:18.108102 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:18.108085 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:27:18.108740 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:18.108720 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:27:20.968791 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:20.968757 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:21.069465 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:21.069434 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:22.602086 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:22.602056 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:22.702733 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:22.702699 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:22.903761 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:22.903633 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:27.163113 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:27.163031 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:32.973740 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:32.973706 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:33.038331 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:33.038267 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:37.811251 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:37.811220 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q"] Jul 14 01:27:37.811699 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:37.811434 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" containerID="cri-o://62f3f1bca1753e05bdae900620657a515cc0e108a246dbb267e7b7f2dfbdae80" gracePeriod=30 Jul 14 01:27:37.840169 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:37.831665 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:38.048502 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:38.048467 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:38.678714 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:38.678676 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:38.780220 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:38.780184 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:39.863472 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:39.863434 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:41.597734 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:41.597701 2557 patch_prober.go:28] interesting pod/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 14 01:27:41.598085 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:41.597759 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 01:27:42.962707 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:42.962669 2557 generic.go:356] "Generic (PLEG): container finished" podID="80d18117-5960-4b16-b4ac-426a94c0020a" containerID="62f3f1bca1753e05bdae900620657a515cc0e108a246dbb267e7b7f2dfbdae80" exitCode=0 Jul 14 01:27:42.963116 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:42.962706 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" event={"ID":"80d18117-5960-4b16-b4ac-426a94c0020a","Type":"ContainerDied","Data":"62f3f1bca1753e05bdae900620657a515cc0e108a246dbb267e7b7f2dfbdae80"} Jul 14 01:27:42.963116 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:42.962770 2557 scope.go:117] "RemoveContainer" containerID="64ea195b96dc1c5fa010eb3b2577566413c94bc9c604e0e078eb0d0cb4d0e899" Jul 14 01:27:43.080131 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.080078 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:27:43.211308 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.211276 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-data\") pod \"80d18117-5960-4b16-b4ac-426a94c0020a\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " Jul 14 01:27:43.211308 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.211312 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2zz6\" (UniqueName: \"kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-kube-api-access-j2zz6\") pod \"80d18117-5960-4b16-b4ac-426a94c0020a\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " Jul 14 01:27:43.211553 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.211343 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-credential-socket\") pod \"80d18117-5960-4b16-b4ac-426a94c0020a\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " Jul 14 01:27:43.211553 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.211531 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80d18117-5960-4b16-b4ac-426a94c0020a-istio-podinfo\") pod \"80d18117-5960-4b16-b4ac-426a94c0020a\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " Jul 14 01:27:43.211668 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.211572 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-istio-token\") pod \"80d18117-5960-4b16-b4ac-426a94c0020a\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " Jul 14 01:27:43.211668 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.211595 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80d18117-5960-4b16-b4ac-426a94c0020a-istiod-ca-cert\") pod \"80d18117-5960-4b16-b4ac-426a94c0020a\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " Jul 14 01:27:43.211668 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.211614 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "80d18117-5960-4b16-b4ac-426a94c0020a" (UID: "80d18117-5960-4b16-b4ac-426a94c0020a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:27:43.211668 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.211651 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-socket\") pod \"80d18117-5960-4b16-b4ac-426a94c0020a\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " Jul 14 01:27:43.211911 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.211681 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-certs\") pod \"80d18117-5960-4b16-b4ac-426a94c0020a\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " Jul 14 01:27:43.211911 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.211707 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-envoy\") pod \"80d18117-5960-4b16-b4ac-426a94c0020a\" (UID: \"80d18117-5960-4b16-b4ac-426a94c0020a\") " Jul 14 01:27:43.212013 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.211918 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "80d18117-5960-4b16-b4ac-426a94c0020a" (UID: "80d18117-5960-4b16-b4ac-426a94c0020a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:27:43.212071 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.212020 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80d18117-5960-4b16-b4ac-426a94c0020a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "80d18117-5960-4b16-b4ac-426a94c0020a" (UID: "80d18117-5960-4b16-b4ac-426a94c0020a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:27:43.212180 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.212110 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "80d18117-5960-4b16-b4ac-426a94c0020a" (UID: "80d18117-5960-4b16-b4ac-426a94c0020a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:27:43.212352 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.212277 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-data" (OuterVolumeSpecName: "istio-data") pod "80d18117-5960-4b16-b4ac-426a94c0020a" (UID: "80d18117-5960-4b16-b4ac-426a94c0020a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:27:43.212469 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.212029 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:27:43.212469 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.212380 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-credential-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:27:43.213684 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.213655 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/80d18117-5960-4b16-b4ac-426a94c0020a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "80d18117-5960-4b16-b4ac-426a94c0020a" (UID: "80d18117-5960-4b16-b4ac-426a94c0020a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:27:43.213767 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.213702 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-kube-api-access-j2zz6" (OuterVolumeSpecName: "kube-api-access-j2zz6") pod "80d18117-5960-4b16-b4ac-426a94c0020a" (UID: "80d18117-5960-4b16-b4ac-426a94c0020a"). InnerVolumeSpecName "kube-api-access-j2zz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:27:43.213814 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.213756 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-istio-token" (OuterVolumeSpecName: "istio-token") pod "80d18117-5960-4b16-b4ac-426a94c0020a" (UID: "80d18117-5960-4b16-b4ac-426a94c0020a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:27:43.213938 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.213920 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "80d18117-5960-4b16-b4ac-426a94c0020a" (UID: "80d18117-5960-4b16-b4ac-426a94c0020a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:27:43.313534 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.313505 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80d18117-5960-4b16-b4ac-426a94c0020a-istio-podinfo\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:27:43.313534 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.313528 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-istio-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:27:43.313534 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.313537 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80d18117-5960-4b16-b4ac-426a94c0020a-istiod-ca-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:27:43.313728 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.313545 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-workload-certs\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:27:43.313728 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.313553 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-envoy\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:27:43.313728 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.313560 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80d18117-5960-4b16-b4ac-426a94c0020a-istio-data\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:27:43.313728 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.313567 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2zz6\" (UniqueName: \"kubernetes.io/projected/80d18117-5960-4b16-b4ac-426a94c0020a-kube-api-access-j2zz6\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:27:43.969639 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.969559 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" Jul 14 01:27:43.970056 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.969558 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q" event={"ID":"80d18117-5960-4b16-b4ac-426a94c0020a","Type":"ContainerDied","Data":"ae4699220b7a721d1443a9c825d8a8cef8c90f67c2cd64d55529eff8c6dca0da"} Jul 14 01:27:43.970056 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.969692 2557 scope.go:117] "RemoveContainer" containerID="62f3f1bca1753e05bdae900620657a515cc0e108a246dbb267e7b7f2dfbdae80" Jul 14 01:27:43.985417 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.985396 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q"] Jul 14 01:27:43.988617 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:43.988595 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-cb7605-openshift-default-6b47898b6f-ndm9q"] Jul 14 01:27:45.679180 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:45.679087 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" path="/var/lib/kubelet/pods/80d18117-5960-4b16-b4ac-426a94c0020a/volumes" Jul 14 01:27:48.639710 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:48.639675 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:53.245999 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:53.245968 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:58.889902 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:58.889867 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:27:58.948870 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:27:58.948828 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:03.947676 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:03.947642 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn"] Jul 14 01:28:03.948092 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:03.947871 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerName="istio-proxy" containerID="cri-o://a46779299f3112b9347436389592dca12faf68f9639d5598ae36634498e9fda6" gracePeriod=30 Jul 14 01:28:03.968660 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:03.968628 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:04.202774 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.202689 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:04.556041 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.556002 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:04.837005 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.836936 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r"] Jul 14 01:28:04.837353 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.837339 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" Jul 14 01:28:04.837408 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.837358 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" Jul 14 01:28:04.837408 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.837377 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" Jul 14 01:28:04.837408 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.837382 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" Jul 14 01:28:04.837505 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.837451 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" Jul 14 01:28:04.837505 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.837463 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="80d18117-5960-4b16-b4ac-426a94c0020a" containerName="istio-proxy" Jul 14 01:28:04.847075 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.846988 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.849053 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.848980 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-99f60e-openshift-default-dockercfg-hj47x\"" Jul 14 01:28:04.851546 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.851484 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r"] Jul 14 01:28:04.859030 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.859003 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:04.884001 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.883975 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-podinfo\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.884106 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.884046 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-envoy\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.884106 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.884086 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-token\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.884230 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.884109 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpnjl\" (UniqueName: \"kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-kube-api-access-lpnjl\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.884230 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.884139 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-socket\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.884230 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.884218 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-certs\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.884347 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.884254 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-credential-socket\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.884347 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.884281 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-data\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.884347 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.884300 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f29708c5-6ec0-445d-b022-6ddab9a99abe-istiod-ca-cert\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.985836 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.985800 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-envoy\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986297 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.985846 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-token\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986297 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.985869 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpnjl\" (UniqueName: \"kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-kube-api-access-lpnjl\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986297 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.985906 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-socket\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986297 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.985932 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-certs\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986297 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.985959 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-credential-socket\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986587 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.986173 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-data\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986587 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.986459 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f29708c5-6ec0-445d-b022-6ddab9a99abe-istiod-ca-cert\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986587 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.986503 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-data\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986587 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.986357 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-credential-socket\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986587 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.986526 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-podinfo\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986587 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.986383 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-socket\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.986587 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.986416 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-certs\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.987036 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.987014 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f29708c5-6ec0-445d-b022-6ddab9a99abe-istiod-ca-cert\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.988486 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.988465 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-envoy\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.988739 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.988721 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-podinfo\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.992601 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.992575 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-token\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:04.992764 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:04.992747 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpnjl\" (UniqueName: \"kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-kube-api-access-lpnjl\") pod \"e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:05.078140 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:05.078109 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:05.159854 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:05.159782 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:05.297007 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:05.296986 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r"] Jul 14 01:28:05.299055 ip-10-0-134-143 kubenswrapper[2557]: W0714 01:28:05.299029 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf29708c5_6ec0_445d_b022_6ddab9a99abe.slice/crio-f804d11734d0eec951b9e051fa10a716261f64677244b521b696fe8c2795ccf2 WatchSource:0}: Error finding container f804d11734d0eec951b9e051fa10a716261f64677244b521b696fe8c2795ccf2: Status 404 returned error can't find the container with id f804d11734d0eec951b9e051fa10a716261f64677244b521b696fe8c2795ccf2 Jul 14 01:28:05.301588 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:05.301554 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:05.301666 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:05.301621 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:05.301666 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:05.301657 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:06.096669 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:06.096630 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" event={"ID":"f29708c5-6ec0-445d-b022-6ddab9a99abe","Type":"ContainerStarted","Data":"24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310"} Jul 14 01:28:06.097033 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:06.096693 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" event={"ID":"f29708c5-6ec0-445d-b022-6ddab9a99abe","Type":"ContainerStarted","Data":"f804d11734d0eec951b9e051fa10a716261f64677244b521b696fe8c2795ccf2"} Jul 14 01:28:06.115206 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:06.115160 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" podStartSLOduration=2.115130921 podStartE2EDuration="2.115130921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:28:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:28:06.113626844 +0000 UTC m=+2622.964994214" watchObservedRunningTime="2026-07-14 01:28:06.115130921 +0000 UTC m=+2622.966498291" Jul 14 01:28:06.159948 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:06.159925 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:06.164769 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:06.164749 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:07.101428 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:07.101402 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:07.102407 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:07.102385 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:08.036902 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:08.036846 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:08.139360 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:08.139328 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:08.239518 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:08.239483 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:08.493211 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:08.493107 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:08.662213 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:08.662180 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:08.762840 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:08.762765 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:09.115539 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.115511 2557 generic.go:356] "Generic (PLEG): container finished" podID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerID="a46779299f3112b9347436389592dca12faf68f9639d5598ae36634498e9fda6" exitCode=0 Jul 14 01:28:09.115727 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.115591 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" event={"ID":"fd537788-34cd-42f5-af19-92bae1b3f98b","Type":"ContainerDied","Data":"a46779299f3112b9347436389592dca12faf68f9639d5598ae36634498e9fda6"} Jul 14 01:28:09.115727 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.115664 2557 scope.go:117] "RemoveContainer" containerID="cef10afc5b1ad74bc40f6a5d1b006b6c4d7fcfd595e6eaf25ace40388ba73f53" Jul 14 01:28:09.206443 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.206423 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:28:09.223121 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.223098 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-data\") pod \"fd537788-34cd-42f5-af19-92bae1b3f98b\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " Jul 14 01:28:09.223306 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.223127 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-podinfo\") pod \"fd537788-34cd-42f5-af19-92bae1b3f98b\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " Jul 14 01:28:09.223306 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.223202 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-token\") pod \"fd537788-34cd-42f5-af19-92bae1b3f98b\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " Jul 14 01:28:09.223306 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.223282 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-credential-socket\") pod \"fd537788-34cd-42f5-af19-92bae1b3f98b\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " Jul 14 01:28:09.223590 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.223309 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-socket\") pod \"fd537788-34cd-42f5-af19-92bae1b3f98b\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " Jul 14 01:28:09.223590 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.223347 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-envoy\") pod \"fd537788-34cd-42f5-af19-92bae1b3f98b\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " Jul 14 01:28:09.223590 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.223388 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd537788-34cd-42f5-af19-92bae1b3f98b-istiod-ca-cert\") pod \"fd537788-34cd-42f5-af19-92bae1b3f98b\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " Jul 14 01:28:09.223590 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.223421 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-certs\") pod \"fd537788-34cd-42f5-af19-92bae1b3f98b\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " Jul 14 01:28:09.223590 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.223449 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27m8t\" (UniqueName: \"kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-kube-api-access-27m8t\") pod \"fd537788-34cd-42f5-af19-92bae1b3f98b\" (UID: \"fd537788-34cd-42f5-af19-92bae1b3f98b\") " Jul 14 01:28:09.223990 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.223914 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-data" (OuterVolumeSpecName: "istio-data") pod "fd537788-34cd-42f5-af19-92bae1b3f98b" (UID: "fd537788-34cd-42f5-af19-92bae1b3f98b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:28:09.225079 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.224837 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fd537788-34cd-42f5-af19-92bae1b3f98b" (UID: "fd537788-34cd-42f5-af19-92bae1b3f98b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:28:09.225079 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.224910 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd537788-34cd-42f5-af19-92bae1b3f98b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fd537788-34cd-42f5-af19-92bae1b3f98b" (UID: "fd537788-34cd-42f5-af19-92bae1b3f98b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:28:09.225433 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.225410 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fd537788-34cd-42f5-af19-92bae1b3f98b" (UID: "fd537788-34cd-42f5-af19-92bae1b3f98b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:28:09.225538 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.225471 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fd537788-34cd-42f5-af19-92bae1b3f98b" (UID: "fd537788-34cd-42f5-af19-92bae1b3f98b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:28:09.226735 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.226705 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-token" (OuterVolumeSpecName: "istio-token") pod "fd537788-34cd-42f5-af19-92bae1b3f98b" (UID: "fd537788-34cd-42f5-af19-92bae1b3f98b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:28:09.226850 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.226796 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-kube-api-access-27m8t" (OuterVolumeSpecName: "kube-api-access-27m8t") pod "fd537788-34cd-42f5-af19-92bae1b3f98b" (UID: "fd537788-34cd-42f5-af19-92bae1b3f98b"). InnerVolumeSpecName "kube-api-access-27m8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:28:09.226914 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.226885 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fd537788-34cd-42f5-af19-92bae1b3f98b" (UID: "fd537788-34cd-42f5-af19-92bae1b3f98b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:28:09.227062 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.227046 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fd537788-34cd-42f5-af19-92bae1b3f98b" (UID: "fd537788-34cd-42f5-af19-92bae1b3f98b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:28:09.324570 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.324519 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-data\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:28:09.324570 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.324542 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-podinfo\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:28:09.324570 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.324551 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:28:09.324570 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.324560 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-credential-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:28:09.324570 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.324569 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:28:09.324801 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.324578 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-istio-envoy\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:28:09.324801 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.324587 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd537788-34cd-42f5-af19-92bae1b3f98b-istiod-ca-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:28:09.324801 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.324595 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd537788-34cd-42f5-af19-92bae1b3f98b-workload-certs\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:28:09.324801 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:09.324602 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27m8t\" (UniqueName: \"kubernetes.io/projected/fd537788-34cd-42f5-af19-92bae1b3f98b-kube-api-access-27m8t\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:28:10.122188 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:10.122131 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" event={"ID":"fd537788-34cd-42f5-af19-92bae1b3f98b","Type":"ContainerDied","Data":"dd0df0d181ef3f38a7c9fb9db410a557255b3139c886f4494d125bf17eb5af47"} Jul 14 01:28:10.122188 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:10.122178 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn" Jul 14 01:28:10.122489 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:10.122212 2557 scope.go:117] "RemoveContainer" containerID="a46779299f3112b9347436389592dca12faf68f9639d5598ae36634498e9fda6" Jul 14 01:28:10.137869 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:10.137846 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn"] Jul 14 01:28:10.140160 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:10.140131 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-e7c477-openshift-default-78486cd679-5pfsn"] Jul 14 01:28:11.679703 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:11.679670 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" path="/var/lib/kubelet/pods/fd537788-34cd-42f5-af19-92bae1b3f98b/volumes" Jul 14 01:28:12.567001 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:12.566968 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:12.913630 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:12.913546 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:13.165538 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.165465 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz"] Jul 14 01:28:13.165852 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.165840 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerName="istio-proxy" Jul 14 01:28:13.165899 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.165854 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerName="istio-proxy" Jul 14 01:28:13.165934 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.165921 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerName="istio-proxy" Jul 14 01:28:13.169388 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.169364 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.171213 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.171195 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-8fd48b-openshift-default-dockercfg-j5nf5\"" Jul 14 01:28:13.179660 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.179639 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz"] Jul 14 01:28:13.189351 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.189319 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:13.260478 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.260448 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/729c1de3-1be4-48b8-85c8-d26c8acc1133-istiod-ca-cert\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.260718 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.260698 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-socket\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.260877 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.260859 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-credential-socket\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.261020 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.261006 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-envoy\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.261169 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.261136 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-podinfo\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.261338 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.261322 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-certs\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.261489 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.261475 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7jbq\" (UniqueName: \"kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-kube-api-access-f7jbq\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.261653 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.261639 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-data\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.261790 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.261774 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-token\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.363351 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.363314 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-certs\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.363546 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.363369 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7jbq\" (UniqueName: \"kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-kube-api-access-f7jbq\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.363546 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.363410 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-data\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.363546 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.363435 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-token\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.363546 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.363480 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/729c1de3-1be4-48b8-85c8-d26c8acc1133-istiod-ca-cert\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.363546 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.363506 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-socket\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.363546 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.363542 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-credential-socket\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.363886 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.363577 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-envoy\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.363886 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.363604 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-podinfo\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.363886 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.363712 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-certs\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.363886 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.363802 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-data\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.364099 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.364000 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-socket\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.364099 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.364084 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-credential-socket\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.364422 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.364370 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/729c1de3-1be4-48b8-85c8-d26c8acc1133-istiod-ca-cert\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.366173 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.366133 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-podinfo\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.366268 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.366194 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-envoy\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.372181 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.372124 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-token\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.372819 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.372690 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7jbq\" (UniqueName: \"kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-kube-api-access-f7jbq\") pod \"e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.484698 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.484624 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:13.626569 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.626533 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz"] Jul 14 01:28:13.628142 ip-10-0-134-143 kubenswrapper[2557]: W0714 01:28:13.628106 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod729c1de3_1be4_48b8_85c8_d26c8acc1133.slice/crio-c2fea38db0127752e1dd70150b2a2851664444cc6c06bb789a2d092cd4668f1e WatchSource:0}: Error finding container c2fea38db0127752e1dd70150b2a2851664444cc6c06bb789a2d092cd4668f1e: Status 404 returned error can't find the container with id c2fea38db0127752e1dd70150b2a2851664444cc6c06bb789a2d092cd4668f1e Jul 14 01:28:13.630386 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.630351 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:13.630455 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.630436 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:13.630497 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:13.630485 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:14.023714 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:14.023679 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:14.147558 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:14.147517 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" event={"ID":"729c1de3-1be4-48b8-85c8-d26c8acc1133","Type":"ContainerStarted","Data":"6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c"} Jul 14 01:28:14.147712 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:14.147573 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" event={"ID":"729c1de3-1be4-48b8-85c8-d26c8acc1133","Type":"ContainerStarted","Data":"c2fea38db0127752e1dd70150b2a2851664444cc6c06bb789a2d092cd4668f1e"} Jul 14 01:28:14.164972 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:14.164925 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" podStartSLOduration=1.164910014 podStartE2EDuration="1.164910014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 01:28:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:28:14.162960242 +0000 UTC m=+2631.014327612" watchObservedRunningTime="2026-07-14 01:28:14.164910014 +0000 UTC m=+2631.016277384" Jul 14 01:28:14.485324 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:14.485289 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:14.489961 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:14.489938 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:15.153885 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:15.153851 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:15.154835 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:15.154813 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:16.080583 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:16.080548 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:16.166327 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:16.166293 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:16.230207 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:16.230168 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:16.519390 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:16.519285 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:16.716360 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:16.716321 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:16.818142 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:16.818107 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:20.595686 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:20.595646 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:22.103886 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:22.103796 2557 patch_prober.go:28] interesting pod/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 14 01:28:22.104371 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:22.103875 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 14 01:28:22.201318 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:22.201223 2557 generic.go:356] "Generic (PLEG): container finished" podID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerID="24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310" exitCode=0 Jul 14 01:28:22.201318 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:22.201302 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" event={"ID":"f29708c5-6ec0-445d-b022-6ddab9a99abe","Type":"ContainerDied","Data":"24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310"} Jul 14 01:28:22.201800 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:22.201784 2557 scope.go:117] "RemoveContainer" containerID="24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310" Jul 14 01:28:22.202847 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:22.202821 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:22.202900 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:22.202887 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:22.202935 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:22.202919 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:23.209363 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:23.209328 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" event={"ID":"f29708c5-6ec0-445d-b022-6ddab9a99abe","Type":"ContainerStarted","Data":"1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6"} Jul 14 01:28:24.160101 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:24.160025 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:24.160254 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:24.160197 2557 patch_prober.go:28] interesting pod/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 14 01:28:24.160254 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:24.160241 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 14 01:28:25.160636 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:25.160602 2557 patch_prober.go:28] interesting pod/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 14 01:28:25.161051 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:25.160655 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 14 01:28:26.161236 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:26.161121 2557 patch_prober.go:28] interesting pod/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 14 01:28:26.161236 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:26.161219 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 14 01:28:27.161201 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:27.161108 2557 patch_prober.go:28] interesting pod/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body= Jul 14 01:28:27.161369 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:27.161200 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" Jul 14 01:28:28.164230 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:28.164198 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:28.164719 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:28.164699 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:28.165313 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:28.165297 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:28:29.252236 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:29.252207 2557 generic.go:356] "Generic (PLEG): container finished" podID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerID="6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c" exitCode=0 Jul 14 01:28:29.252647 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:29.252286 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" event={"ID":"729c1de3-1be4-48b8-85c8-d26c8acc1133","Type":"ContainerDied","Data":"6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c"} Jul 14 01:28:29.252841 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:29.252824 2557 scope.go:117] "RemoveContainer" containerID="6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c" Jul 14 01:28:29.253913 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:29.253887 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:29.253976 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:29.253954 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:29.254012 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:29.254002 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 14 01:28:29.448170 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:29.447898 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:30.108308 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:30.108264 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:30.154090 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:30.154060 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:30.168573 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:30.168545 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:30.259529 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:30.259489 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" event={"ID":"729c1de3-1be4-48b8-85c8-d26c8acc1133","Type":"ContainerStarted","Data":"08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e"} Jul 14 01:28:30.484913 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:30.484814 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:30.485982 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:30.485958 2557 patch_prober.go:28] interesting pod/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 14 01:28:30.486097 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:30.486014 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 14 01:28:31.485217 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:31.485172 2557 patch_prober.go:28] interesting pod/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 14 01:28:31.485705 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:31.485240 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 14 01:28:32.486118 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:32.486079 2557 patch_prober.go:28] interesting pod/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 14 01:28:32.486748 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:32.486712 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 14 01:28:33.485414 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:33.485373 2557 patch_prober.go:28] interesting pod/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 14 01:28:33.485709 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:33.485678 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 14 01:28:34.486037 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:34.485998 2557 patch_prober.go:28] interesting pod/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body= Jul 14 01:28:34.486632 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:34.486598 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" Jul 14 01:28:35.489045 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:35.489013 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:35.489462 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:35.489424 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:35.490016 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:35.489998 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:28:38.275162 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:38.275096 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:28:39.639748 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:28:39.639714 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:29:04.047250 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:29:04.047205 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:29:05.418903 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:29:05.418858 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:29:23.811661 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:29:23.811635 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:29:23.825586 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:29:23.825567 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:30:10.015791 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:30:10.015751 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:30:10.116394 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:30:10.116367 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:30:47.802885 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:30:47.802809 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:31:02.653724 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:31:02.653683 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:31:03.624538 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:31:03.624501 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:31:13.548364 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:31:13.548328 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:31:23.998887 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:31:23.998851 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:31:24.101304 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:31:24.101267 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:31:40.864894 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:31:40.864851 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:32:17.030957 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:32:17.030917 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:32:18.158419 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:32:18.158378 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:32:39.443054 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:32:39.443021 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:32:43.747005 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:32:43.746969 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:32:43.848927 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:32:43.848888 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:33:17.061401 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:33:17.061366 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:33:18.943230 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:33:18.943184 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:33:30.808042 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:33:30.808006 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:33:30.908457 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:33:30.908422 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:33:39.107749 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:33:39.107712 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:33:39.207791 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:33:39.207747 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:33:39.438374 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:33:39.438290 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:33:57.568833 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:33:57.568746 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:33:58.754098 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:33:58.754065 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:17.725329 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:17.725289 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:17.825650 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:17.825615 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:23.847017 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:23.846985 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:34:23.863531 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:23.863508 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:34:27.657739 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:27.657692 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:32.329478 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:32.329439 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:38.246776 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:38.246740 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:38.347324 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:38.347287 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:43.010186 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:43.010130 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r"] Jul 14 01:34:43.010640 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:43.010425 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerName="istio-proxy" containerID="cri-o://1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6" gracePeriod=30 Jul 14 01:34:43.030919 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:43.030885 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:43.166641 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:43.166608 2557 patch_prober.go:28] interesting pod/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 01:34:43.166826 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:43.166698 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 01:34:43.261872 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:43.261795 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:43.857760 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:43.857728 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:43.961092 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:43.961064 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:48.260997 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.260972 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:34:48.285130 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285107 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f29708c5-6ec0-445d-b022-6ddab9a99abe-istiod-ca-cert\") pod \"f29708c5-6ec0-445d-b022-6ddab9a99abe\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " Jul 14 01:34:48.285282 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285140 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpnjl\" (UniqueName: \"kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-kube-api-access-lpnjl\") pod \"f29708c5-6ec0-445d-b022-6ddab9a99abe\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " Jul 14 01:34:48.285282 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285183 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-credential-socket\") pod \"f29708c5-6ec0-445d-b022-6ddab9a99abe\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " Jul 14 01:34:48.285282 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285242 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-podinfo\") pod \"f29708c5-6ec0-445d-b022-6ddab9a99abe\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " Jul 14 01:34:48.285457 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285284 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-certs\") pod \"f29708c5-6ec0-445d-b022-6ddab9a99abe\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " Jul 14 01:34:48.285457 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285328 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-token\") pod \"f29708c5-6ec0-445d-b022-6ddab9a99abe\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " Jul 14 01:34:48.285556 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285452 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f29708c5-6ec0-445d-b022-6ddab9a99abe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f29708c5-6ec0-445d-b022-6ddab9a99abe" (UID: "f29708c5-6ec0-445d-b022-6ddab9a99abe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:34:48.285556 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285509 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f29708c5-6ec0-445d-b022-6ddab9a99abe" (UID: "f29708c5-6ec0-445d-b022-6ddab9a99abe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:34:48.285663 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285609 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f29708c5-6ec0-445d-b022-6ddab9a99abe" (UID: "f29708c5-6ec0-445d-b022-6ddab9a99abe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:34:48.285710 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285657 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-envoy\") pod \"f29708c5-6ec0-445d-b022-6ddab9a99abe\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " Jul 14 01:34:48.285764 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285707 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-data\") pod \"f29708c5-6ec0-445d-b022-6ddab9a99abe\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " Jul 14 01:34:48.285811 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.285773 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-socket\") pod \"f29708c5-6ec0-445d-b022-6ddab9a99abe\" (UID: \"f29708c5-6ec0-445d-b022-6ddab9a99abe\") " Jul 14 01:34:48.286381 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.286183 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f29708c5-6ec0-445d-b022-6ddab9a99abe-istiod-ca-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:34:48.286381 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.286211 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-credential-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:34:48.286381 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.286226 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-certs\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:34:48.286695 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.286657 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f29708c5-6ec0-445d-b022-6ddab9a99abe" (UID: "f29708c5-6ec0-445d-b022-6ddab9a99abe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:34:48.287052 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.287027 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-data" (OuterVolumeSpecName: "istio-data") pod "f29708c5-6ec0-445d-b022-6ddab9a99abe" (UID: "f29708c5-6ec0-445d-b022-6ddab9a99abe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:34:48.287887 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.287861 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-kube-api-access-lpnjl" (OuterVolumeSpecName: "kube-api-access-lpnjl") pod "f29708c5-6ec0-445d-b022-6ddab9a99abe" (UID: "f29708c5-6ec0-445d-b022-6ddab9a99abe"). InnerVolumeSpecName "kube-api-access-lpnjl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:34:48.288039 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.288012 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f29708c5-6ec0-445d-b022-6ddab9a99abe" (UID: "f29708c5-6ec0-445d-b022-6ddab9a99abe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:34:48.288111 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.288030 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-token" (OuterVolumeSpecName: "istio-token") pod "f29708c5-6ec0-445d-b022-6ddab9a99abe" (UID: "f29708c5-6ec0-445d-b022-6ddab9a99abe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:34:48.288627 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.288598 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f29708c5-6ec0-445d-b022-6ddab9a99abe" (UID: "f29708c5-6ec0-445d-b022-6ddab9a99abe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:34:48.342298 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.342270 2557 generic.go:356] "Generic (PLEG): container finished" podID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerID="1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6" exitCode=0 Jul 14 01:34:48.342392 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.342324 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" Jul 14 01:34:48.342392 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.342359 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" event={"ID":"f29708c5-6ec0-445d-b022-6ddab9a99abe","Type":"ContainerDied","Data":"1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6"} Jul 14 01:34:48.342467 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.342416 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r" event={"ID":"f29708c5-6ec0-445d-b022-6ddab9a99abe","Type":"ContainerDied","Data":"f804d11734d0eec951b9e051fa10a716261f64677244b521b696fe8c2795ccf2"} Jul 14 01:34:48.342467 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.342441 2557 scope.go:117] "RemoveContainer" containerID="1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6" Jul 14 01:34:48.351781 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.351765 2557 scope.go:117] "RemoveContainer" containerID="24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310" Jul 14 01:34:48.360206 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.360189 2557 scope.go:117] "RemoveContainer" containerID="1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6" Jul 14 01:34:48.360521 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:34:48.360492 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6\": container with ID starting with 1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6 not found: ID does not exist" containerID="1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6" Jul 14 01:34:48.360615 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.360523 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6"} err="failed to get container status \"1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6\": rpc error: code = NotFound desc = could not find container \"1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6\": container with ID starting with 1580266fd75e45ffd8b2118fcf287f5d00bb54d52307c3bdb8355ddd7592abb6 not found: ID does not exist" Jul 14 01:34:48.360615 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.360547 2557 scope.go:117] "RemoveContainer" containerID="24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310" Jul 14 01:34:48.360831 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:34:48.360809 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310\": container with ID starting with 24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310 not found: ID does not exist" containerID="24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310" Jul 14 01:34:48.360877 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.360841 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310"} err="failed to get container status \"24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310\": rpc error: code = NotFound desc = could not find container \"24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310\": container with ID starting with 24fb11833cd5e6e4564b225f8d3a623a5d27957cf88801ed8ee40570ccc38310 not found: ID does not exist" Jul 14 01:34:48.362514 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.362496 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r"] Jul 14 01:34:48.366193 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.366128 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-99f60e-openshift-default-b679d9f95-xqs2r"] Jul 14 01:34:48.387121 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.387099 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-podinfo\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:34:48.387121 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.387118 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:34:48.387254 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.387127 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-envoy\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:34:48.387254 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.387136 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-istio-data\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:34:48.387254 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.387143 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f29708c5-6ec0-445d-b022-6ddab9a99abe-workload-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:34:48.387254 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:48.387174 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpnjl\" (UniqueName: \"kubernetes.io/projected/f29708c5-6ec0-445d-b022-6ddab9a99abe-kube-api-access-lpnjl\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:34:49.677687 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:49.677651 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" path="/var/lib/kubelet/pods/f29708c5-6ec0-445d-b022-6ddab9a99abe/volumes" Jul 14 01:34:53.793131 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:53.793095 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:34:58.459108 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:34:58.459071 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:35:04.278514 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:04.278472 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:35:04.377532 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:04.377493 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:35:09.119647 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:09.119611 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz"] Jul 14 01:35:09.120099 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:09.119832 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" containerID="cri-o://08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e" gracePeriod=30 Jul 14 01:35:09.140544 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:09.140489 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:35:09.372849 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:09.372773 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:35:09.828525 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:09.828492 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-95cf4bc4b-mhrkc_aa0c837b-56c9-43a9-a055-718c2d2c08a6/authorino/0.log" Jul 14 01:35:14.368173 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.368131 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:35:14.489985 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.489923 2557 generic.go:356] "Generic (PLEG): container finished" podID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerID="08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e" exitCode=0 Jul 14 01:35:14.489985 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.489964 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" Jul 14 01:35:14.489985 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.489967 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" event={"ID":"729c1de3-1be4-48b8-85c8-d26c8acc1133","Type":"ContainerDied","Data":"08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e"} Jul 14 01:35:14.490212 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.490008 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz" event={"ID":"729c1de3-1be4-48b8-85c8-d26c8acc1133","Type":"ContainerDied","Data":"c2fea38db0127752e1dd70150b2a2851664444cc6c06bb789a2d092cd4668f1e"} Jul 14 01:35:14.490212 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.490030 2557 scope.go:117] "RemoveContainer" containerID="08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e" Jul 14 01:35:14.494496 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494474 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-credential-socket\") pod \"729c1de3-1be4-48b8-85c8-d26c8acc1133\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " Jul 14 01:35:14.494624 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494517 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-envoy\") pod \"729c1de3-1be4-48b8-85c8-d26c8acc1133\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " Jul 14 01:35:14.494624 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494552 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/729c1de3-1be4-48b8-85c8-d26c8acc1133-istiod-ca-cert\") pod \"729c1de3-1be4-48b8-85c8-d26c8acc1133\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " Jul 14 01:35:14.494624 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494568 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-socket\") pod \"729c1de3-1be4-48b8-85c8-d26c8acc1133\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " Jul 14 01:35:14.494799 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494698 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7jbq\" (UniqueName: \"kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-kube-api-access-f7jbq\") pod \"729c1de3-1be4-48b8-85c8-d26c8acc1133\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " Jul 14 01:35:14.494799 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494749 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-data\") pod \"729c1de3-1be4-48b8-85c8-d26c8acc1133\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " Jul 14 01:35:14.494799 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494750 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "729c1de3-1be4-48b8-85c8-d26c8acc1133" (UID: "729c1de3-1be4-48b8-85c8-d26c8acc1133"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:35:14.494952 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494804 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-certs\") pod \"729c1de3-1be4-48b8-85c8-d26c8acc1133\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " Jul 14 01:35:14.494952 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494827 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-token\") pod \"729c1de3-1be4-48b8-85c8-d26c8acc1133\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " Jul 14 01:35:14.494952 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494842 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "729c1de3-1be4-48b8-85c8-d26c8acc1133" (UID: "729c1de3-1be4-48b8-85c8-d26c8acc1133"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:35:14.494952 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494875 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-podinfo\") pod \"729c1de3-1be4-48b8-85c8-d26c8acc1133\" (UID: \"729c1de3-1be4-48b8-85c8-d26c8acc1133\") " Jul 14 01:35:14.494952 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.494944 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/729c1de3-1be4-48b8-85c8-d26c8acc1133-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "729c1de3-1be4-48b8-85c8-d26c8acc1133" (UID: "729c1de3-1be4-48b8-85c8-d26c8acc1133"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 01:35:14.495234 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.495192 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-credential-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:35:14.495234 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.495209 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/729c1de3-1be4-48b8-85c8-d26c8acc1133-istiod-ca-cert\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:35:14.495234 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.495213 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "729c1de3-1be4-48b8-85c8-d26c8acc1133" (UID: "729c1de3-1be4-48b8-85c8-d26c8acc1133"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:35:14.495234 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.495222 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-socket\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:35:14.495574 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.495553 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-data" (OuterVolumeSpecName: "istio-data") pod "729c1de3-1be4-48b8-85c8-d26c8acc1133" (UID: "729c1de3-1be4-48b8-85c8-d26c8acc1133"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:35:14.496979 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.496951 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-token" (OuterVolumeSpecName: "istio-token") pod "729c1de3-1be4-48b8-85c8-d26c8acc1133" (UID: "729c1de3-1be4-48b8-85c8-d26c8acc1133"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:35:14.497217 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.497195 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-kube-api-access-f7jbq" (OuterVolumeSpecName: "kube-api-access-f7jbq") pod "729c1de3-1be4-48b8-85c8-d26c8acc1133" (UID: "729c1de3-1be4-48b8-85c8-d26c8acc1133"). InnerVolumeSpecName "kube-api-access-f7jbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 01:35:14.497289 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.497265 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "729c1de3-1be4-48b8-85c8-d26c8acc1133" (UID: "729c1de3-1be4-48b8-85c8-d26c8acc1133"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 01:35:14.497494 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.497473 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "729c1de3-1be4-48b8-85c8-d26c8acc1133" (UID: "729c1de3-1be4-48b8-85c8-d26c8acc1133"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 01:35:14.499667 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.499636 2557 scope.go:117] "RemoveContainer" containerID="6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c" Jul 14 01:35:14.514932 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.514918 2557 scope.go:117] "RemoveContainer" containerID="08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e" Jul 14 01:35:14.515183 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:35:14.515164 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e\": container with ID starting with 08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e not found: ID does not exist" containerID="08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e" Jul 14 01:35:14.515242 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.515191 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e"} err="failed to get container status \"08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e\": rpc error: code = NotFound desc = could not find container \"08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e\": container with ID starting with 08ab32c59be573b22a247ca4f8b8118ff97c25908caa9804bf44aae66de2e03e not found: ID does not exist" Jul 14 01:35:14.515242 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.515208 2557 scope.go:117] "RemoveContainer" containerID="6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c" Jul 14 01:35:14.515442 ip-10-0-134-143 kubenswrapper[2557]: E0714 01:35:14.515421 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c\": container with ID starting with 6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c not found: ID does not exist" containerID="6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c" Jul 14 01:35:14.515483 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.515448 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c"} err="failed to get container status \"6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c\": rpc error: code = NotFound desc = could not find container \"6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c\": container with ID starting with 6c43306abbb107fa4b55c4495d6b3e695f71071159e410fb9c0628ab7b66a40c not found: ID does not exist" Jul 14 01:35:14.596300 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.596276 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7jbq\" (UniqueName: \"kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-kube-api-access-f7jbq\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:35:14.596300 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.596296 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-data\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:35:14.596426 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.596308 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-workload-certs\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:35:14.596426 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.596316 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-token\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:35:14.596426 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.596324 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-podinfo\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:35:14.596426 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.596332 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/729c1de3-1be4-48b8-85c8-d26c8acc1133-istio-envoy\") on node \"ip-10-0-134-143.ec2.internal\" DevicePath \"\"" Jul 14 01:35:14.765629 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.765563 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gkcmr_db690295-f070-4975-ad18-bbb2e599affb/manager/0.log" Jul 14 01:35:14.810842 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.810821 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz"] Jul 14 01:35:14.814134 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.814115 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-8fd48b-openshift-default-554f549595-dxrnz"] Jul 14 01:35:14.897143 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:14.897119 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bsdwz_8a592437-0608-4a2a-9e28-09ab2395b241/manager/0.log" Jul 14 01:35:15.145412 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:15.145360 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c8skl_ea299afa-3425-445e-817f-469542f41680/server/0.log" Jul 14 01:35:15.274388 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:15.274368 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kjxkv_18bbc1d0-0722-40c4-b6aa-c400cc1882ed/manager/0.log" Jul 14 01:35:15.394930 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:15.394905 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b7d6985f7-25whd_0a2ecbb3-980b-4ad5-a899-a92616131a09/copy-manifests/0.log" Jul 14 01:35:15.400921 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:15.400869 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b7d6985f7-25whd_0a2ecbb3-980b-4ad5-a899-a92616131a09/manager/0.log" Jul 14 01:35:15.678121 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:15.678050 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" path="/var/lib/kubelet/pods/729c1de3-1be4-48b8-85c8-d26c8acc1133/volumes" Jul 14 01:35:18.696486 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:18.696459 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php_db1c7be4-2ea5-4cb0-9dd0-e440bae11050/util/0.log" Jul 14 01:35:18.701363 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:18.701339 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php_db1c7be4-2ea5-4cb0-9dd0-e440bae11050/pull/0.log" Jul 14 01:35:18.705908 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:18.705884 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php_db1c7be4-2ea5-4cb0-9dd0-e440bae11050/extract/0.log" Jul 14 01:35:18.818954 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:18.818916 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz_257eede3-e32d-4320-bf9d-b1b2f95b118d/util/0.log" Jul 14 01:35:18.824925 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:18.824894 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz_257eede3-e32d-4320-bf9d-b1b2f95b118d/pull/0.log" Jul 14 01:35:18.829772 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:18.829753 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz_257eede3-e32d-4320-bf9d-b1b2f95b118d/extract/0.log" Jul 14 01:35:18.940715 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:18.940691 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb_55f5baad-9e1e-478f-8ba1-4d1dc53a8527/util/0.log" Jul 14 01:35:18.945921 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:18.945906 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb_55f5baad-9e1e-478f-8ba1-4d1dc53a8527/pull/0.log" Jul 14 01:35:18.950809 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:18.950751 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb_55f5baad-9e1e-478f-8ba1-4d1dc53a8527/extract/0.log" Jul 14 01:35:19.068621 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:19.068601 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj_b1008438-9427-4355-83c6-e1084301e7dd/pull/0.log" Jul 14 01:35:19.076403 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:19.076384 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj_b1008438-9427-4355-83c6-e1084301e7dd/extract/0.log" Jul 14 01:35:19.081165 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:19.081135 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj_b1008438-9427-4355-83c6-e1084301e7dd/util/0.log" Jul 14 01:35:19.207918 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:19.207857 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-95cf4bc4b-mhrkc_aa0c837b-56c9-43a9-a055-718c2d2c08a6/authorino/0.log" Jul 14 01:35:19.323319 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:19.323302 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-g7mht_b6e80bb6-828a-4dda-9a70-0bda2c7ac0b3/manager/0.log" Jul 14 01:35:19.672325 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:19.672302 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-n7rkg_3cb21005-3f32-45ac-ae4c-5b862656a112/registry-server/0.log" Jul 14 01:35:19.807324 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:19.807301 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-swm4n_5944e5ac-0819-45fa-b8c3-871a0a110e34/manager/0.log" Jul 14 01:35:20.047015 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:20.046993 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cvgq5_28802c53-86c9-4c60-94ec-66daf134046c/manager/0.log" Jul 14 01:35:20.418722 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:20.418659 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wz6h5_9d1d8abe-5a15-48cd-a639-5141082cc001/istio-proxy/0.log" Jul 14 01:35:20.650466 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:20.650434 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f876ccc94-tgk5r_a656bf6b-2f6c-4e6d-aee0-998cb9aafc22/kube-auth-proxy/0.log" Jul 14 01:35:20.883323 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:20.883299 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hxshd_ad7dd378-440e-4ae4-b524-197073a5b229/istio-proxy/1.log" Jul 14 01:35:21.130889 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:21.130860 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-jlqgh_faf77f07-737b-4772-961a-4fdfd8432cea/payload-processing/0.log" Jul 14 01:35:21.725475 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:21.725446 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vgnzz_3978cfae-8977-4685-9e59-c1c3d1659fda/main/0.log" Jul 14 01:35:21.838643 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:21.838617 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-vzhj4_c95978e5-862d-4e87-ac1b-5cf1e949e6f4/main/0.log" Jul 14 01:35:22.194342 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:22.194309 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-tcms8_a53bd28d-b04a-4b49-8a2e-f1ab9c206d38/main/0.log" Jul 14 01:35:26.259783 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:26.259751 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:35:26.326496 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:26.326467 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" limit=1024 Jul 14 01:35:28.954248 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:28.954218 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nzt74_5d75eb52-5424-4c0e-a6bc-1fe4d642587a/global-pull-secret-syncer/0.log" Jul 14 01:35:29.068047 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:29.068016 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-9dwkf_b4febc6f-5f93-4d99-bf2c-1a4211db67f8/konnectivity-agent/0.log" Jul 14 01:35:29.208161 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:29.208065 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-143.ec2.internal_68f0c074f7c50b5b863d7294622d4fc6/haproxy/0.log" Jul 14 01:35:33.878368 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:33.878338 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php_db1c7be4-2ea5-4cb0-9dd0-e440bae11050/extract/0.log" Jul 14 01:35:33.906042 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:33.906017 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php_db1c7be4-2ea5-4cb0-9dd0-e440bae11050/util/0.log" Jul 14 01:35:33.928046 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:33.928019 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759s9php_db1c7be4-2ea5-4cb0-9dd0-e440bae11050/pull/0.log" Jul 14 01:35:33.960357 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:33.960331 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz_257eede3-e32d-4320-bf9d-b1b2f95b118d/extract/0.log" Jul 14 01:35:33.986991 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:33.986970 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz_257eede3-e32d-4320-bf9d-b1b2f95b118d/util/0.log" Jul 14 01:35:34.011122 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:34.011094 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0rvngz_257eede3-e32d-4320-bf9d-b1b2f95b118d/pull/0.log" Jul 14 01:35:34.034384 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:34.034362 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb_55f5baad-9e1e-478f-8ba1-4d1dc53a8527/extract/0.log" Jul 14 01:35:34.051789 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:34.051769 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb_55f5baad-9e1e-478f-8ba1-4d1dc53a8527/util/0.log" Jul 14 01:35:34.070543 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:34.070524 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vbrpb_55f5baad-9e1e-478f-8ba1-4d1dc53a8527/pull/0.log" Jul 14 01:35:34.097174 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:34.097139 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj_b1008438-9427-4355-83c6-e1084301e7dd/extract/0.log" Jul 14 01:35:34.121387 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:34.121367 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj_b1008438-9427-4355-83c6-e1084301e7dd/util/0.log" Jul 14 01:35:34.139342 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:34.139278 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mgqxj_b1008438-9427-4355-83c6-e1084301e7dd/pull/0.log" Jul 14 01:35:34.702513 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:34.702479 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-95cf4bc4b-mhrkc_aa0c837b-56c9-43a9-a055-718c2d2c08a6/authorino/0.log" Jul 14 01:35:34.738896 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:34.738870 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-g7mht_b6e80bb6-828a-4dda-9a70-0bda2c7ac0b3/manager/0.log" Jul 14 01:35:34.832619 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:34.832589 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-n7rkg_3cb21005-3f32-45ac-ae4c-5b862656a112/registry-server/0.log" Jul 14 01:35:34.991732 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:34.991632 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-swm4n_5944e5ac-0819-45fa-b8c3-871a0a110e34/manager/0.log" Jul 14 01:35:35.119894 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:35.119857 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cvgq5_28802c53-86c9-4c60-94ec-66daf134046c/manager/0.log" Jul 14 01:35:36.902864 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:36.902823 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-28k6t_0daf5dc4-d3ee-4414-8858-3e83d3ba2abf/kube-state-metrics/0.log" Jul 14 01:35:36.919720 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:36.919698 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-28k6t_0daf5dc4-d3ee-4414-8858-3e83d3ba2abf/kube-rbac-proxy-main/0.log" Jul 14 01:35:36.939074 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:36.939056 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-28k6t_0daf5dc4-d3ee-4414-8858-3e83d3ba2abf/kube-rbac-proxy-self/0.log" Jul 14 01:35:37.004090 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:37.004062 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5dpnm_081b053a-2396-4f00-8f3d-e867400806bb/node-exporter/0.log" Jul 14 01:35:37.020979 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:37.020955 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5dpnm_081b053a-2396-4f00-8f3d-e867400806bb/kube-rbac-proxy/0.log" Jul 14 01:35:37.039545 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:37.039524 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-5dpnm_081b053a-2396-4f00-8f3d-e867400806bb/init-textfile/0.log" Jul 14 01:35:38.763204 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763176 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw"] Jul 14 01:35:38.763582 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763564 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerName="istio-proxy" Jul 14 01:35:38.763582 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763578 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerName="istio-proxy" Jul 14 01:35:38.763680 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763590 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" Jul 14 01:35:38.763680 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763595 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" Jul 14 01:35:38.763680 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763619 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerName="istio-proxy" Jul 14 01:35:38.763680 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763628 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerName="istio-proxy" Jul 14 01:35:38.763680 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763643 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" Jul 14 01:35:38.763680 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763669 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" Jul 14 01:35:38.763963 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763759 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" Jul 14 01:35:38.763963 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763774 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd537788-34cd-42f5-af19-92bae1b3f98b" containerName="istio-proxy" Jul 14 01:35:38.763963 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763783 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="729c1de3-1be4-48b8-85c8-d26c8acc1133" containerName="istio-proxy" Jul 14 01:35:38.763963 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763793 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerName="istio-proxy" Jul 14 01:35:38.763963 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.763803 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f29708c5-6ec0-445d-b022-6ddab9a99abe" containerName="istio-proxy" Jul 14 01:35:38.766358 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.766339 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.770471 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.770451 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nwk5x\"/\"kube-root-ca.crt\"" Jul 14 01:35:38.770740 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.770723 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nwk5x\"/\"openshift-service-ca.crt\"" Jul 14 01:35:38.771086 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.771071 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nwk5x\"/\"default-dockercfg-zsgk7\"" Jul 14 01:35:38.779555 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.779533 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw"] Jul 14 01:35:38.895082 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.895051 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-sys\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.895082 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.895084 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-proc\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.895337 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.895114 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-podres\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.895337 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.895229 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-lib-modules\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.895337 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.895280 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fjs9\" (UniqueName: \"kubernetes.io/projected/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-kube-api-access-4fjs9\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.996517 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.996483 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-sys\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.996517 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.996516 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-proc\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.996725 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.996545 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-podres\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.996725 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.996606 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-sys\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.996725 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.996607 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-lib-modules\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.996725 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.996672 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fjs9\" (UniqueName: \"kubernetes.io/projected/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-kube-api-access-4fjs9\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.996725 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.996614 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-proc\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.996725 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.996714 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-lib-modules\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:38.996725 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:38.996714 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-podres\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:39.003447 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:39.003424 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fjs9\" (UniqueName: \"kubernetes.io/projected/a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f-kube-api-access-4fjs9\") pod \"perf-node-gather-daemonset-vtwdw\" (UID: \"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f\") " pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:39.076438 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:39.076416 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:39.204858 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:39.204832 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw"] Jul 14 01:35:39.206640 ip-10-0-134-143 kubenswrapper[2557]: W0714 01:35:39.206612 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda99ffd5d_2c93_4cb6_91e7_3ca78bd2d55f.slice/crio-1d7647232e3432b3fa1c178384fb7390e54e1bd80239e70955bcbf96913de885 WatchSource:0}: Error finding container 1d7647232e3432b3fa1c178384fb7390e54e1bd80239e70955bcbf96913de885: Status 404 returned error can't find the container with id 1d7647232e3432b3fa1c178384fb7390e54e1bd80239e70955bcbf96913de885 Jul 14 01:35:39.622425 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:39.622392 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" event={"ID":"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f","Type":"ContainerStarted","Data":"790c7f557ab8579280cbe391864f379eaf5dc896f7e110de3bbb01343ad57b99"} Jul 14 01:35:39.622634 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:39.622441 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" event={"ID":"a99ffd5d-2c93-4cb6-91e7-3ca78bd2d55f","Type":"ContainerStarted","Data":"1d7647232e3432b3fa1c178384fb7390e54e1bd80239e70955bcbf96913de885"} Jul 14 01:35:39.622634 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:39.622521 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:39.636830 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:39.636783 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" podStartSLOduration=1.636770108 podStartE2EDuration="1.636770108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 01:35:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 01:35:39.634401654 +0000 UTC m=+3076.485769035" watchObservedRunningTime="2026-07-14 01:35:39.636770108 +0000 UTC m=+3076.488137477" Jul 14 01:35:39.761994 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:39.761965 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-595f89b4d9-rvrbn_01e70466-94a2-429f-b650-e59133fa738d/console/0.log" Jul 14 01:35:39.798025 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:39.798002 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-lpkgf_f0354a6b-116e-4fa2-b001-0dd0434a2ec1/download-server/0.log" Jul 14 01:35:41.327334 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:41.327299 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-74hmz_47b07188-e3fa-4065-8026-9715a5b6cb41/dns/0.log" Jul 14 01:35:41.345597 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:41.345569 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-74hmz_47b07188-e3fa-4065-8026-9715a5b6cb41/kube-rbac-proxy/0.log" Jul 14 01:35:41.446892 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:41.446866 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7xct5_c8dacd2e-6336-4956-aa9f-e9a0964534d0/dns-node-resolver/0.log" Jul 14 01:35:41.986213 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:41.986182 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6f6b674cc4-wxvnx_5020942f-9d6a-400f-8714-6da764b5e6ea/registry/0.log" Jul 14 01:35:42.028855 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:42.028833 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qqpzf_427427dc-5248-4fdb-b49f-1309f98f2320/node-ca/0.log" Jul 14 01:35:42.950265 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:42.950238 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wz6h5_9d1d8abe-5a15-48cd-a639-5141082cc001/istio-proxy/0.log" Jul 14 01:35:43.179483 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:43.179450 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f876ccc94-tgk5r_a656bf6b-2f6c-4e6d-aee0-998cb9aafc22/kube-auth-proxy/0.log" Jul 14 01:35:43.225578 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:43.225492 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hxshd_ad7dd378-440e-4ae4-b524-197073a5b229/istio-proxy/0.log" Jul 14 01:35:43.283425 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:43.283401 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hxshd_ad7dd378-440e-4ae4-b524-197073a5b229/istio-proxy/1.log" Jul 14 01:35:43.356036 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:43.356008 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-jlqgh_faf77f07-737b-4772-961a-4fdfd8432cea/payload-processing/0.log" Jul 14 01:35:44.036658 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:44.036618 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-7n47x_95b6216a-8104-4322-960f-f53bd026111f/serve-healthcheck-canary/0.log" Jul 14 01:35:44.514413 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:44.514341 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-hbcm5_c824f147-b37c-41d9-a199-3f1399bb93d5/insights-operator/0.log" Jul 14 01:35:44.517286 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:44.517266 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-hbcm5_c824f147-b37c-41d9-a199-3f1399bb93d5/insights-operator/1.log" Jul 14 01:35:44.537163 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:44.537137 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m7464_847c344a-b356-4dbc-8143-d7f3154fdf3a/kube-rbac-proxy/0.log" Jul 14 01:35:44.554408 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:44.554388 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m7464_847c344a-b356-4dbc-8143-d7f3154fdf3a/exporter/0.log" Jul 14 01:35:44.573520 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:44.573502 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m7464_847c344a-b356-4dbc-8143-d7f3154fdf3a/extractor/0.log" Jul 14 01:35:45.637165 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:45.637123 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nwk5x/perf-node-gather-daemonset-vtwdw" Jul 14 01:35:46.662101 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:46.662071 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-gkcmr_db690295-f070-4975-ad18-bbb2e599affb/manager/0.log" Jul 14 01:35:46.899133 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:46.899099 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bsdwz_8a592437-0608-4a2a-9e28-09ab2395b241/manager/0.log" Jul 14 01:35:47.105901 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:47.105875 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c8skl_ea299afa-3425-445e-817f-469542f41680/server/0.log" Jul 14 01:35:47.207452 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:47.207428 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-kjxkv_18bbc1d0-0722-40c4-b6aa-c400cc1882ed/manager/0.log" Jul 14 01:35:47.228779 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:47.228758 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b7d6985f7-25whd_0a2ecbb3-980b-4ad5-a899-a92616131a09/manager/0.log" Jul 14 01:35:47.245205 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:47.245178 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b7d6985f7-25whd_0a2ecbb3-980b-4ad5-a899-a92616131a09/copy-manifests/0.log" Jul 14 01:35:49.379505 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:49.379483 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wz6h5_9d1d8abe-5a15-48cd-a639-5141082cc001/istio-proxy/0.log" Jul 14 01:35:49.587339 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:49.587302 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f876ccc94-tgk5r_a656bf6b-2f6c-4e6d-aee0-998cb9aafc22/kube-auth-proxy/0.log" Jul 14 01:35:49.633393 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:49.633371 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hxshd_ad7dd378-440e-4ae4-b524-197073a5b229/istio-proxy/0.log" Jul 14 01:35:49.698613 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:49.698578 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hxshd_ad7dd378-440e-4ae4-b524-197073a5b229/istio-proxy/1.log" Jul 14 01:35:49.754021 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:49.753995 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-jlqgh_faf77f07-737b-4772-961a-4fdfd8432cea/payload-processing/0.log" Jul 14 01:35:51.008313 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:51.008283 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wz6h5_9d1d8abe-5a15-48cd-a639-5141082cc001/istio-proxy/0.log" Jul 14 01:35:51.221361 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:51.221336 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5f876ccc94-tgk5r_a656bf6b-2f6c-4e6d-aee0-998cb9aafc22/kube-auth-proxy/0.log" Jul 14 01:35:51.263777 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:51.263707 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hxshd_ad7dd378-440e-4ae4-b524-197073a5b229/istio-proxy/0.log" Jul 14 01:35:51.323002 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:51.322977 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-hxshd_ad7dd378-440e-4ae4-b524-197073a5b229/istio-proxy/1.log" Jul 14 01:35:51.382979 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:51.382951 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-jlqgh_faf77f07-737b-4772-961a-4fdfd8432cea/payload-processing/0.log" Jul 14 01:35:55.256563 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:55.256536 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njjqg_afc9e28a-2764-4e91-884b-ead5986764cc/kube-multus-additional-cni-plugins/0.log" Jul 14 01:35:55.273720 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:55.273700 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njjqg_afc9e28a-2764-4e91-884b-ead5986764cc/egress-router-binary-copy/0.log" Jul 14 01:35:55.290443 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:55.290424 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njjqg_afc9e28a-2764-4e91-884b-ead5986764cc/cni-plugins/0.log" Jul 14 01:35:55.306814 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:55.306793 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njjqg_afc9e28a-2764-4e91-884b-ead5986764cc/bond-cni-plugin/0.log" Jul 14 01:35:55.323296 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:55.323269 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njjqg_afc9e28a-2764-4e91-884b-ead5986764cc/routeoverride-cni/0.log" Jul 14 01:35:55.339744 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:55.339724 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njjqg_afc9e28a-2764-4e91-884b-ead5986764cc/whereabouts-cni-bincopy/0.log" Jul 14 01:35:55.356034 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:55.356019 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-njjqg_afc9e28a-2764-4e91-884b-ead5986764cc/whereabouts-cni/0.log" Jul 14 01:35:55.395296 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:55.395274 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-frmbq_cea48c24-3755-41c7-b30c-5d60aa682341/kube-multus/0.log" Jul 14 01:35:55.527128 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:55.527075 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-txzdm_4f92c479-1507-4bd3-a7a3-0bcdb89a049f/network-metrics-daemon/0.log" Jul 14 01:35:55.542751 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:55.542732 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-txzdm_4f92c479-1507-4bd3-a7a3-0bcdb89a049f/kube-rbac-proxy/0.log" Jul 14 01:35:56.809420 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:56.809389 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-controller/0.log" Jul 14 01:35:56.823043 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:56.823015 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/0.log" Jul 14 01:35:56.823448 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:56.823428 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovn-acl-logging/1.log" Jul 14 01:35:56.845258 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:56.845240 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/kube-rbac-proxy-node/0.log" Jul 14 01:35:56.868276 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:56.868256 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/kube-rbac-proxy-ovn-metrics/0.log" Jul 14 01:35:56.882918 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:56.882895 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/northd/0.log" Jul 14 01:35:56.899878 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:56.899854 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/nbdb/0.log" Jul 14 01:35:56.918906 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:56.918887 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/sbdb/0.log" Jul 14 01:35:57.002989 ip-10-0-134-143 kubenswrapper[2557]: I0714 01:35:57.002965 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-v9t68_2bae43db-11b2-47ff-951c-604be5df51cd/ovnkube-controller/0.log"