Jul 10 02:05:34.388261 ip-10-0-131-213 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 10 02:05:34.388276 ip-10-0-131-213 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 10 02:05:34.388284 ip-10-0-131-213 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 10 02:05:34.388717 ip-10-0-131-213 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 10 02:05:44.610669 ip-10-0-131-213 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 10 02:05:44.610686 ip-10-0-131-213 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 4940ef26614241f9bc6d3e7a99c06e40 --
Jul 10 02:07:23.560113 ip-10-0-131-213 systemd[1]: Starting Kubernetes Kubelet...
Jul 10 02:07:23.996861 ip-10-0-131-213 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:07:23.996861 ip-10-0-131-213 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 10 02:07:23.996861 ip-10-0-131-213 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:07:23.996861 ip-10-0-131-213 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 10 02:07:23.996861 ip-10-0-131-213 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 10 02:07:23.999015 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:23.998933    2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 10 02:07:24.004933 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004917    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:07:24.004933 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004933    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004939    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004944    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004947    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004950    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004953    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004956    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004959    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004962    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004964    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004967    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004970    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004973    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004975    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004978    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004981    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004984    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004994    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004997    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:07:24.004995 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.004999    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005002    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005005    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005030    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005034    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005037    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005041    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005045    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005048    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005050    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005053    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005056    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005059    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005062    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005065    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005068    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005071    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005073    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005077    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005081    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:07:24.005454 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005084    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005088    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005090    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005093    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005095    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005098    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005100    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005103    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005105    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005108    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005110    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005113    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005115    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005117    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005121    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005124    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005127    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005129    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005132    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:07:24.005955 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005134    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005137    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005139    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005142    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005145    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005147    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005150    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005153    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005155    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005158    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005160    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005163    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005166    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005168    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005170    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005173    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005175    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005178    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005180    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:07:24.006428 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005184    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005187    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005189    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005192    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005194    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005196    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005199    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005201    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005206    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005209    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005211    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005214    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005216    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005219    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005221    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005225    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005228    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005231    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005233    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005236    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:07:24.006897 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005238    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005947    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005954    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005958    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005961    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005964    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005967    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005970    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005973    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005976    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005979    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005981    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005983    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005986    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005988    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005991    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005993    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005996    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.005999    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006002    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:07:24.007395 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006005    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006008    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006010    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006013    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006016    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006018    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006020    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006023    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006025    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006028    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006030    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006033    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006035    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006038    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006040    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006043    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006045    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006049    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006053    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:07:24.007958 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006057    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006060    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006064    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006067    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006070    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006073    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006075    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006078    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006080    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006083    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006086    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006089    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006092    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006095    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006097    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006100    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006102    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006105    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006107    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006110    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:07:24.008415 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006112    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006115    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006118    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006120    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006123    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006125    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006129    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006131    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006134    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006136    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006139    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006141    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006144    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006147    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006149    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006152    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006154    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006157    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006159    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006162    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:07:24.008907 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006164    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006166    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006169    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006172    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006175    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006178    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006180    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006183    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006186    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006188    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006191    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006193    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006196    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006198    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006201    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006204    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006206    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006208    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006213    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:07:24.009382 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006216    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006219    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006570    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006574    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006577    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006580    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006584    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006587    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006589    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006592    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006594    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006597    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006599    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006602    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006605    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006608    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006611    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006614    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006617    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:07:24.009852 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006620    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006622    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006625    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006627    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006630    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006633    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006635    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006638    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006640    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006643    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006645    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006648    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006650    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006653    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006655    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006658    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006661    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006665    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006668    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:07:24.010300 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006671    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006674    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006677    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006680    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006683    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006685    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006688    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006691    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006693    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006696    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006699    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006703    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006707    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006710    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006713    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006716    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006719    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006721    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006723    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:07:24.010755 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006726    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006728    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006731    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006733    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006736    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006738    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006741    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006743    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006746    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006748    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006750    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006753    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006755    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006758    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006760    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006763    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006766    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006768    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006771    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006773    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:07:24.011233 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006776    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006778    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006780    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006783    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006786    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006789    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006792    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006794    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006797    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006815    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006818    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006821    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006823    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006826    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006828    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006831    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006833    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006836    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006838    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006841    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:07:24.011721 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006843    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006846    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006848    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.006851    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006922    2557 flags.go:64] FLAG: --address="0.0.0.0"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006929    2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006934    2557 flags.go:64] FLAG: --anonymous-auth="true"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006939    2557 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006944    2557 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006947    2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006951    2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006955    2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006958    2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006961    2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006964    2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006968    2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006971    2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006974    2557 flags.go:64] FLAG: --cgroup-root=""
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006978    2557 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006981    2557 flags.go:64] FLAG: --client-ca-file=""
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006985    2557 flags.go:64] FLAG: --cloud-provider="external"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006987    2557 flags.go:64] FLAG: --cluster-dns="[]"
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006993    2557 flags.go:64] FLAG: --cluster-domain=""
Jul 10 02:07:24.012207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006996    2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.006999    2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007002    2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007005    2557 flags.go:64] FLAG: --container-log-max-files="5"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007009    2557 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007012    2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007015    2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007018    2557 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007021    2557 flags.go:64] FLAG: --contention-profiling="false"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007024    2557 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007027    2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007030    2557 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007032    2557 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007036    2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007039    2557 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007042    2557 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007045    2557 flags.go:64] FLAG: --enable-load-reader="false"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007048    2557 flags.go:64] FLAG: --enable-server="true"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007052    2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007056    2557 flags.go:64] FLAG: --event-burst="100"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007059    2557 flags.go:64] FLAG: --event-qps="50"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007062    2557 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007065    2557 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007068    2557 flags.go:64] FLAG: --eviction-hard=""
Jul 10 02:07:24.012748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007073    2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007076    2557 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007081    2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007084    2557 flags.go:64] FLAG: --eviction-soft=""
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007087    2557 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007090    2557 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007093    2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007096    2557 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007099    2557 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007101    2557 flags.go:64] FLAG: --fail-swap-on="true"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007104    2557 flags.go:64] FLAG: --feature-gates=""
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007108    2557 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007111    2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007114    2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007117    2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007120    2557 flags.go:64] FLAG: --healthz-port="10248"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007123    2557 flags.go:64] FLAG: --help="false"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007126    2557 flags.go:64] FLAG: --hostname-override="ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007129    2557 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007132    2557 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007135    2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007138    2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007141    2557 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007144    2557 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 10 02:07:24.013411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007147    2557 flags.go:64] FLAG: --image-service-endpoint=""
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007150    2557 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007153    2557 flags.go:64] FLAG: --kube-api-burst="100"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007156    2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007159    2557 flags.go:64] FLAG: --kube-api-qps="50"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007162    2557 flags.go:64] FLAG: --kube-reserved=""
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007165    2557 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007168    2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007173    2557 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007176    2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007181    2557 flags.go:64] FLAG: --lock-file=""
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007184    2557 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007187    2557 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007190    2557 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007195    2557 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007198    2557 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007201    2557 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007204    2557 flags.go:64] FLAG: --logging-format="text"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007206    2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007210    2557 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007213    2557 flags.go:64] FLAG: --manifest-url=""
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007215    2557 flags.go:64] FLAG: --manifest-url-header=""
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007220    2557 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007223    2557 flags.go:64] FLAG: --max-open-files="1000000"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007227    2557 flags.go:64] FLAG: --max-pods="110"
Jul 10 02:07:24.014102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007230    2557 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007233    2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007236    2557 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007238    2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007241    2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007244    2557 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007247    2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007253    2557 flags.go:64] FLAG: --node-status-max-images="50"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007256    2557 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007259    2557 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007262    2557 flags.go:64] FLAG: --pod-cidr=""
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007266    2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007271    2557 flags.go:64] FLAG: --pod-manifest-path=""
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007274    2557 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007277    2557 flags.go:64] FLAG: --pods-per-core="0"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007280    2557 flags.go:64] FLAG: --port="10250"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007284    2557 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007289    2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d67abbafa168ccc9"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007292    2557 flags.go:64] FLAG: --qos-reserved=""
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007295    2557 flags.go:64] FLAG: --read-only-port="10255"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007298    2557 flags.go:64] FLAG: --register-node="true"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007301    2557 flags.go:64] FLAG: --register-with-taints=""
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007304    2557 flags.go:64] FLAG: --registry-burst="10"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007307    2557 flags.go:64] FLAG: --registry-qps="5"
Jul 10 02:07:24.015163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007310    2557 flags.go:64] FLAG: --reserved-cpus=""
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007313    2557 flags.go:64] FLAG: --reserved-memory=""
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007316    2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007319    2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007322    2557 flags.go:64] FLAG: --rotate-certificates="false"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007325    2557 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007328    2557 flags.go:64] FLAG: --runonce="false"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007331    2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007333    2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007336    2557 flags.go:64] FLAG: --seccomp-default="false"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007339    2557 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007342    2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007345    2557 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007348    2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007351    2557 flags.go:64] FLAG: --storage-driver-password="root"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007354    2557 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007357    2557 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007360    2557 flags.go:64] FLAG: --storage-driver-user="root"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007363    2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007366    2557 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007369    2557 flags.go:64] FLAG: --system-cgroups=""
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007372    2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007377    2557 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007380    2557 flags.go:64] FLAG: --tls-cert-file=""
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007384    2557 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 10 02:07:24.016199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007390    2557 flags.go:64] FLAG: --tls-min-version=""
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007392    2557 flags.go:64] FLAG: --tls-private-key-file=""
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007396    2557 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007399    2557 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007402    2557 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007405    2557 flags.go:64] FLAG: --v="2"
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007409    2557 flags.go:64] FLAG: --version="false"
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007413    2557 flags.go:64] FLAG: --vmodule=""
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007417    2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.007420    2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007536    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007539    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007542    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007545    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007548    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007550    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007554    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007558    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007561    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007564    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007567    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:07:24.017275 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007570    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007573    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007575    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007578    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007581    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007583    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007586    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007590    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007592    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007595    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007611    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007616    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007619    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007622    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007624    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007628    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007632    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007634    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007637    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007640    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:07:24.018014 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007642    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007645    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007648    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007650    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007653    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007656    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007658    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007661    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007663    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007666    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007668    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007671    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007673    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007676    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007679    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007681    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007684    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007686    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007688    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:07:24.018642 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007691    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007694    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007697    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007701    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007705    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007708    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007710    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007713    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007716    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007718    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007721    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007723    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007726    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007728    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007731    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007734    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007736    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007739    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007741    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007744    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:07:24.019488 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007746    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007749    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007751    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007754    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007756    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007759    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007761    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007764    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007767    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007769    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007771    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007774    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007777    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007779    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007782    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007786    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007790    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007792    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007795    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007797    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:07:24.020338 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007813    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007815    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007818    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007820    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007823    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007825    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007828    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007832    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.007835    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.008567    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.016242    2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.016260    2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.016286    2557 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.016295    2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 10 02:07:24.020884 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016379    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016387    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016393    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016399    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016404    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016409    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016413    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016417    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016422    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016426    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016431    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016435    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016441    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016445    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016449    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016455    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016459    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016463    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016467    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016473    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:07:24.021237 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016480    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016484    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016489    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016493    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016498    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016502    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016506    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016510    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016515    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016519    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016523    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016528    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016532    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016537    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016541    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016545    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016549    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016554    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016558    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016562    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:07:24.021751 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016567    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016571    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016576    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016580    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016584    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016590    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016594    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016598    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016603    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016607    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016611    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016615    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016619    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016623    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016628    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016632    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016636    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016640    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016645    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:07:24.022269 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016649    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016653    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016658    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016662    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016666    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016670    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016675    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016681    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016687    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016692    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016696    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016701    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016705    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016709    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016713    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016717    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016721    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016725    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016729    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:07:24.022710 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016735    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016739    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016743    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016747    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016752    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016758    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016763    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016768    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016773    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016778    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016782    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016787    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016791    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016796    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016816    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016821    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016826    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016830    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016834    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:07:24.023198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016839    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.016843    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.016851    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017031    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017039    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017044    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017048    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017053    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017057    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017061    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017065    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017069    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017073    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017077    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017083    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 10 02:07:24.023661 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017087    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017091    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017095    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017099    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017103    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017107    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017112    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017116    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017121    2557 feature_gate.go:328] unrecognized feature gate: Example2
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017125    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017130    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017134    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017138    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017142    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017146    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017150    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017155    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017159    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017163    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017167    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 10 02:07:24.024152 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017171    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017175    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017180    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017184    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017188    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017192    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017196    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017200    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017204    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017208    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017212    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017217    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017227    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017231    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017236    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017240    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017244    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017248    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017253    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017257    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 10 02:07:24.024631 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017261    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017265    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017269    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017273    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017277    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017281    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017285    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017288    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017292    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017297    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017301    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017305    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017310    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017314    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017318    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017323    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017327    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017331    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017335    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017339    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 10 02:07:24.025121 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017343    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017347    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017351    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017355    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017359    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017364    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017369    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017373    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017378    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017382    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017386    2557 feature_gate.go:328] unrecognized feature gate: Example
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017390    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017396    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017404    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017411    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017417    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017423    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017428    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 10 02:07:24.025595 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017433    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017437    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017441    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017446    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017451    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017455    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017459    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017463    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:24.017467    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.017475    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.017619    2557 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.022836    2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.024466    2557 server.go:1019] "Starting client certificate rotation"
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.024569    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 02:07:24.026049 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.024609    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 10 02:07:24.027335 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.027321    2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 02:07:24.031424 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.031407    2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 10 02:07:24.044535 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.044517    2557 log.go:25] "Validated CRI v1 runtime API"
Jul 10 02:07:24.047534 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.047513    2557 log.go:25] "Validated CRI v1 image API"
Jul 10 02:07:24.048375 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.048357    2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 10 02:07:24.050959 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.050939    2557 fs.go:135] Filesystem UUIDs: map[6554b981-511c-4711-a798-17521b9b353e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c61e3f31-1c48-4675-927a-7ad82249f8ec:/dev/nvme0n1p4]
Jul 10 02:07:24.051062 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.050958    2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 10 02:07:24.056123 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.056009    2557 manager.go:216] Machine: {Timestamp:2026-07-10 02:07:24.054586401 +0000 UTC m=+0.370896284 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100412 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec261218032f1b46e5a7dfcd4612b53b SystemUUID:ec261218-032f-1b46-e5a7-dfcd4612b53b BootID:4940ef26-6142-41f9-bc6d-3e7a99c06e40 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fd:0f:0e:c2:a1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fd:0f:0e:c2:a1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:7e:cf:c1:33:b6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 10 02:07:24.056123 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.056116    2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 10 02:07:24.056287 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.056225    2557 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 10 02:07:24.059428 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.059412    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 02:07:24.059743 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.059714    2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 10 02:07:24.059942 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.059753    2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-213.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 10 02:07:24.061401 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.061390    2557 topology_manager.go:138] "Creating topology manager with none policy"
Jul 10 02:07:24.061433 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.061403    2557 container_manager_linux.go:309] "Creating device plugin manager"
Jul 10 02:07:24.061433 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.061411    2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 02:07:24.062230 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.062220    2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 10 02:07:24.064143 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.064133    2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 10 02:07:24.068701 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.068689    2557 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 02:07:24.068857 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.068847    2557 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 10 02:07:24.068904 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.068877    2557 kubelet.go:486] "Attempting to sync node with API server"
Jul 10 02:07:24.068904 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.068893    2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 10 02:07:24.068990 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.068911    2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 10 02:07:24.068990 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.068920    2557 kubelet.go:398] "Adding apiserver pod source"
Jul 10 02:07:24.068990 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.068929    2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 10 02:07:24.070377 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.070350    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 02:07:24.070452 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.070395    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 10 02:07:24.073228 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.073210    2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 10 02:07:24.075860 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.075846    2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 10 02:07:24.075914 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.075876    2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 10 02:07:24.076026 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076016    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 10 02:07:24.076061 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076031    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 10 02:07:24.076089 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076036    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 10 02:07:24.076089 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076080    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 10 02:07:24.076143 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076091    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 10 02:07:24.076143 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076096    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 10 02:07:24.076663 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076652    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 10 02:07:24.076695 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076663    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 10 02:07:24.076695 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076671    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 10 02:07:24.076695 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076677    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 10 02:07:24.076695 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076685    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 10 02:07:24.076695 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.076692    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 10 02:07:24.077614 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.077604    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 10 02:07:24.077614 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.077614    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 10 02:07:24.080588 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.080574    2557 server.go:1268] "Started kubelet"
Jul 10 02:07:24.080720 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.080668    2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 10 02:07:24.080815 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.080743    2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 10 02:07:24.080862 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.080841    2557 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 10 02:07:24.081212 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.081192    2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-213.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 10 02:07:24.081288 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.081276    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 10 02:07:24.081427 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.081407    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 10 02:07:24.081472 ip-10-0-131-213 systemd[1]: Started Kubernetes Kubelet.
Jul 10 02:07:24.082398 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.082380    2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 10 02:07:24.082819 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.082791    2557 server.go:310] "Adding debug handlers to kubelet server"
Jul 10 02:07:24.086707 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.086689    2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 10 02:07:24.090677 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.090662    2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 10 02:07:24.090677 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.090677    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 10 02:07:24.090830 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.090761    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-213.ec2.internal\" not found"
Jul 10 02:07:24.091446 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.091431    2557 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 10 02:07:24.091532 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.091456    2557 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 10 02:07:24.091532 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.091430    2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 10 02:07:24.091709 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.091686    2557 reconstruct.go:97] "Volume reconstruction finished"
Jul 10 02:07:24.091709 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.091711    2557 reconciler.go:29] "Reconciler: start to sync state"
Jul 10 02:07:24.091879 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.090520    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-213.ec2.internal.18c0cb898d50c556  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-213.ec2.internal,UID:ip-10-0-131-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-213.ec2.internal,},FirstTimestamp:2026-07-10 02:07:24.08055535 +0000 UTC m=+0.396865214,LastTimestamp:2026-07-10 02:07:24.08055535 +0000 UTC m=+0.396865214,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-213.ec2.internal,}"
Jul 10 02:07:24.091974 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.091959    2557 factory.go:55] Registering systemd factory
Jul 10 02:07:24.091974 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.091976    2557 factory.go:223] Registration of the systemd container factory successfully
Jul 10 02:07:24.092213 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.092200    2557 factory.go:153] Registering CRI-O factory
Jul 10 02:07:24.092213 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.092215    2557 factory.go:223] Registration of the crio container factory successfully
Jul 10 02:07:24.092312 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.092257    2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 10 02:07:24.092312 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.092280    2557 factory.go:103] Registering Raw factory
Jul 10 02:07:24.092312 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.092294    2557 manager.go:1196] Started watching for new ooms in manager
Jul 10 02:07:24.092685 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.092671    2557 manager.go:349] Starting recovery of all containers
Jul 10 02:07:24.095526 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.095484    2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-213.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 10 02:07:24.095643 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.095498    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 10 02:07:24.108488 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.108468    2557 manager.go:354] Recovery completed
Jul 10 02:07:24.113045 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.113032    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:07:24.113504 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.113489    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qjmss"
Jul 10 02:07:24.114918 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.114905    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:07:24.114957 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.114936    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:07:24.114957 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.114950    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:07:24.115553 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.115533    2557 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 10 02:07:24.115553 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.115543    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:07:24.115553 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.115551    2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 10 02:07:24.115694 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.115570    2557 state_mem.go:36] "Initialized new in-memory state store"
Jul 10 02:07:24.116318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.116302    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:07:24.116376 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.116326    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:07:24.116376 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.116341    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:07:24.116552 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.116469    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-213.ec2.internal.18c0cb898f5d1e0f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-213.ec2.internal,UID:ip-10-0-131-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-213.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-213.ec2.internal,},FirstTimestamp:2026-07-10 02:07:24.114918927 +0000 UTC m=+0.431228789,LastTimestamp:2026-07-10 02:07:24.114918927 +0000 UTC m=+0.431228789,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-213.ec2.internal,}"
Jul 10 02:07:24.117770 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.117758    2557 policy_none.go:49] "None policy: Start"
Jul 10 02:07:24.117829 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.117779    2557 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 10 02:07:24.117829 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.117790    2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 10 02:07:24.119720 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.119710    2557 policy_none.go:47] "Start"
Jul 10 02:07:24.121572 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.121554    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qjmss"
Jul 10 02:07:24.127426 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.127361    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-213.ec2.internal.18c0cb898f5d766c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-213.ec2.internal,UID:ip-10-0-131-213.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-213.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-213.ec2.internal,},FirstTimestamp:2026-07-10 02:07:24.114941548 +0000 UTC m=+0.431251412,LastTimestamp:2026-07-10 02:07:24.114941548 +0000 UTC m=+0.431251412,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-213.ec2.internal,}"
Jul 10 02:07:24.175637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.158416    2557 manager.go:337] "Starting Device Plugin manager"
Jul 10 02:07:24.175637 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.158634    2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 10 02:07:24.175637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.158648    2557 server.go:87] "Starting device plugin registration server"
Jul 10 02:07:24.175637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.158876    2557 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 10 02:07:24.175637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.158887    2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 10 02:07:24.175637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.158980    2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 10 02:07:24.175637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.159059    2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 10 02:07:24.175637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.159067    2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 10 02:07:24.175637 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.159671    2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 10 02:07:24.175637 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.159710    2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-213.ec2.internal\" not found"
Jul 10 02:07:24.234187 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.234145    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 10 02:07:24.235378 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.235360    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 10 02:07:24.235452 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.235403    2557 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 10 02:07:24.235452 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.235430    2557 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 10 02:07:24.235536 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.235471    2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 10 02:07:24.238125 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.238109    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:07:24.259177 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.259112    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:07:24.260114 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.260096    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:07:24.260199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.260128    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:07:24.260199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.260140    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:07:24.260199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.260173    2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.272292 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.272271    2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.272369 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.272295    2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-213.ec2.internal\": node \"ip-10-0-131-213.ec2.internal\" not found"
Jul 10 02:07:24.292111 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.292084    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-213.ec2.internal\" not found"
Jul 10 02:07:24.336255 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.336225    2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-213.ec2.internal"]
Jul 10 02:07:24.336368 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.336300    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:07:24.338088 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.338073    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:07:24.338160 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.338101    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:07:24.338160 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.338111    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:07:24.340418 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.340406    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:07:24.340543 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.340527    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.340576 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.340560    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:07:24.341332 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.341308    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:07:24.341332 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.341337    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:07:24.341466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.341347    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:07:24.341466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.341313    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:07:24.341466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.341409    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:07:24.341466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.341419    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:07:24.343546 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.343531    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.343610 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.343557    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 10 02:07:24.344300 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.344284    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientMemory"
Jul 10 02:07:24.344387 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.344308    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasNoDiskPressure"
Jul 10 02:07:24.344387 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.344323    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeHasSufficientPID"
Jul 10 02:07:24.369385 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.369366    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-213.ec2.internal\" not found" node="ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.373568 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.373553    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-213.ec2.internal\" not found" node="ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.392551 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.392528    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-213.ec2.internal\" not found"
Jul 10 02:07:24.393682 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.393666    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b060fe845b58018afc31fc696c483a8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal\" (UID: \"3b060fe845b58018afc31fc696c483a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.393752 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.393695    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0d70d9ea477f5082aeebc1d283121de-config\") pod \"kube-apiserver-proxy-ip-10-0-131-213.ec2.internal\" (UID: \"d0d70d9ea477f5082aeebc1d283121de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.393752 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.393720    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3b060fe845b58018afc31fc696c483a8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal\" (UID: \"3b060fe845b58018afc31fc696c483a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.493381 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.493350    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-213.ec2.internal\" not found"
Jul 10 02:07:24.494473 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.494454    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3b060fe845b58018afc31fc696c483a8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal\" (UID: \"3b060fe845b58018afc31fc696c483a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.494550 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.494509    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3b060fe845b58018afc31fc696c483a8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal\" (UID: \"3b060fe845b58018afc31fc696c483a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.494550 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.494542    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b060fe845b58018afc31fc696c483a8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal\" (UID: \"3b060fe845b58018afc31fc696c483a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.494637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.494565    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0d70d9ea477f5082aeebc1d283121de-config\") pod \"kube-apiserver-proxy-ip-10-0-131-213.ec2.internal\" (UID: \"d0d70d9ea477f5082aeebc1d283121de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.494637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.494567    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3b060fe845b58018afc31fc696c483a8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal\" (UID: \"3b060fe845b58018afc31fc696c483a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.494637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.494590    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d0d70d9ea477f5082aeebc1d283121de-config\") pod \"kube-apiserver-proxy-ip-10-0-131-213.ec2.internal\" (UID: \"d0d70d9ea477f5082aeebc1d283121de\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.594238 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.594162    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-213.ec2.internal\" not found"
Jul 10 02:07:24.674366 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.674334    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.676720 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:24.676694    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:24.695102 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.695080    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-213.ec2.internal\" not found"
Jul 10 02:07:24.795483 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.795435    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-213.ec2.internal\" not found"
Jul 10 02:07:24.898680 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.898599    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-213.ec2.internal\" not found"
Jul 10 02:07:24.998684 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:24.998651    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-213.ec2.internal\" not found"
Jul 10 02:07:25.001388 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.001370    2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:07:25.023820 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.023778    2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 10 02:07:25.023967 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.023916    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 02:07:25.024009 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.023965    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 10 02:07:25.071530 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.071491    2557 apiserver.go:52] "Watching apiserver"
Jul 10 02:07:25.087690 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.087665    2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 10 02:07:25.088532 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.088497    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-8w4fn","openshift-multus/multus-additional-cni-plugins-7z79d","openshift-network-diagnostics/network-check-target-st5kh","openshift-network-operator/iptables-alerter-8zxrq","openshift-cluster-node-tuning-operator/tuned-jmm5j","openshift-dns/node-resolver-4gmwh","openshift-multus/multus-qjjm5","openshift-multus/network-metrics-daemon-5j8nl","openshift-ovn-kubernetes/ovnkube-node-r4vzs","kube-system/global-pull-secret-syncer-mb7lt","kube-system/konnectivity-agent-p7ftq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"]
Jul 10 02:07:25.090831 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.090808    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 10 02:07:25.090958 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.090941    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:25.091395 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.091373    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8w4fn"
Jul 10 02:07:25.093260 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.093242    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dbjd8\""
Jul 10 02:07:25.093631 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.093617    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 10 02:07:25.093717 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.093701    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.094020 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.094002    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 10 02:07:25.094127 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.094113    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 10 02:07:25.095360 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.095346    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 10 02:07:25.095561 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.095542    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 10 02:07:25.095607 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.095588    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 10 02:07:25.095958 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.095945    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:25.096014 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.096001    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 10 02:07:25.096080 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.096037    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:25.096140 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.096119    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2vqc6\""
Jul 10 02:07:25.096198 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.096153    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 10 02:07:25.098216 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.098201    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8zxrq"
Jul 10 02:07:25.099019 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099002    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e7c563a-d986-46de-994c-230d665e028b-host\") pod \"node-ca-8w4fn\" (UID: \"6e7c563a-d986-46de-994c-230d665e028b\") " pod="openshift-image-registry/node-ca-8w4fn"
Jul 10 02:07:25.099091 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099028    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-os-release\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.099091 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099047    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aa9591a-74d2-4890-be3f-482e458bd651-cni-binary-copy\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.099091 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099063    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g62zk\" (UniqueName: \"kubernetes.io/projected/8aa9591a-74d2-4890-be3f-482e458bd651-kube-api-access-g62zk\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.099091 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099084    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6e7c563a-d986-46de-994c-230d665e028b-serviceca\") pod \"node-ca-8w4fn\" (UID: \"6e7c563a-d986-46de-994c-230d665e028b\") " pod="openshift-image-registry/node-ca-8w4fn"
Jul 10 02:07:25.099253 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099128    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cmf9\" (UniqueName: \"kubernetes.io/projected/6e7c563a-d986-46de-994c-230d665e028b-kube-api-access-6cmf9\") pod \"node-ca-8w4fn\" (UID: \"6e7c563a-d986-46de-994c-230d665e028b\") " pod="openshift-image-registry/node-ca-8w4fn"
Jul 10 02:07:25.099253 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099177    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-system-cni-dir\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.099253 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099204    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-cnibin\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.099253 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099229    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.099418 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099252    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8aa9591a-74d2-4890-be3f-482e458bd651-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.099418 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099278    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8aa9591a-74d2-4890-be3f-482e458bd651-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.099918 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099899    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 10 02:07:25.099994 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099898    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:07:25.099994 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.099965    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6lp6p\""
Jul 10 02:07:25.100132 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.100116    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 10 02:07:25.100510 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.100497    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.101956 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.101937    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:07:25.102049 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.102027    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hblbw\""
Jul 10 02:07:25.102477 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.102460    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 10 02:07:25.102833 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.102800    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4gmwh"
Jul 10 02:07:25.104513 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.104489    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 10 02:07:25.104598 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.104529    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 10 02:07:25.104598 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.104575    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qf5w9\""
Jul 10 02:07:25.105068 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.105053    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 10 02:07:25.105348 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.105332    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.107319 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.106929    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-x95k8\""
Jul 10 02:07:25.107740 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.107723    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:25.107740 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.107735    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 10 02:07:25.107870 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.107792    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:25.110168 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.110152    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.110685 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.110668    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 02:07:25.110757 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.110734    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-213.ec2.internal"
Jul 10 02:07:25.112276 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.112243    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 10 02:07:25.112490 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.112472    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 10 02:07:25.114273 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.112651    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 10 02:07:25.114273 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.112966    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 10 02:07:25.114273 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.113259    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x42mc\""
Jul 10 02:07:25.114273 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.113432    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 10 02:07:25.114273 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.113616    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 10 02:07:25.114273 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.113914    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:25.114273 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.113974    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:25.115117 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.115099    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:25.116677 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.116660    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 10 02:07:25.116932 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.116914    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-z7w5m\""
Jul 10 02:07:25.117070 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.117059    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 10 02:07:25.117264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.117247    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal"]
Jul 10 02:07:25.117339 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.117328    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.124132 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.124113    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 10 02:07:25.124823 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.124768    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-131-213.ec2.internal"]
Jul 10 02:07:25.125778 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.125756    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 10 02:07:25.125883 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.125776    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 10 02:07:25.126019 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.125948    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 02:02:25 +0000 UTC" deadline="2028-02-25 10:24:40.071901554 +0000 UTC"
Jul 10 02:07:25.126019 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.126016    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14288h17m14.945888855s"
Jul 10 02:07:25.126178 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.126127    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 10 02:07:25.126178 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.126170    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-z5f8w\""
Jul 10 02:07:25.172409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.172386    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:07:25.192274 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.192258    2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 10 02:07:25.199868 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.199849    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.199962 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.199876    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-dbus\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:25.199962 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.199903    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:25.199962 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.199926    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsqgn\" (UniqueName: \"kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn\") pod \"network-check-target-st5kh\" (UID: \"a85578ca-e7d0-4036-b2c0-8315bf972f66\") " pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:25.199962 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.199941    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-daemon-config\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.200113 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.199964    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.200113 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.199986    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-sys-fs\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.200113 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200008    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-systemd\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.200113 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200033    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-run\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.200113 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200049    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9dc8c62b-0158-46c7-8972-acf60187a450-etc-tuned\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.200113 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200071    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:25.200113 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200096    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xvfh\" (UniqueName: \"kubernetes.io/projected/1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5-kube-api-access-7xvfh\") pod \"iptables-alerter-8zxrq\" (UID: \"1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5\") " pod="openshift-network-operator/iptables-alerter-8zxrq"
Jul 10 02:07:25.200349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200117    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-kubelet\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.200349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200132    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-run-netns\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.200349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200176    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-run-systemd\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.200349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200192    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-run-ovn-kubernetes\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.200349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200207    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8pqn\" (UniqueName: \"kubernetes.io/projected/cbeea170-d687-476c-b932-01800b00ecae-kube-api-access-t8pqn\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.200349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200223    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-sys\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.200349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200246    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-systemd-units\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.200349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200273    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-cni-bin\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.200349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200289    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-socket-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.200349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200322    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2lxt\" (UniqueName: \"kubernetes.io/projected/c541937e-9bfd-48fe-a99f-c97f0e4780f6-kube-api-access-p2lxt\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.200349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200344    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200363    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gbg5\" (UniqueName: \"kubernetes.io/projected/9dc8c62b-0158-46c7-8972-acf60187a450-kube-api-access-6gbg5\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200379    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddrk4\" (UniqueName: \"kubernetes.io/projected/1a790021-08d0-4e49-8da8-f046861c72ac-kube-api-access-ddrk4\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200400    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7ae93db-a16c-4e3b-bb52-59ccc91e033a-konnectivity-ca\") pod \"konnectivity-agent-p7ftq\" (UID: \"b7ae93db-a16c-4e3b-bb52-59ccc91e033a\") " pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200441    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8aa9591a-74d2-4890-be3f-482e458bd651-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200457    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-socket-dir-parent\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200508    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-modprobe-d\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200536    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-kubelet-config\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200558    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trztf\" (UniqueName: \"kubernetes.io/projected/98184b4a-870b-47be-a682-e1f7d43effb0-kube-api-access-trztf\") pod \"node-resolver-4gmwh\" (UID: \"98184b4a-870b-47be-a682-e1f7d43effb0\") " pod="openshift-dns/node-resolver-4gmwh"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200596    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-system-cni-dir\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200617    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-slash\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200637    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cmf9\" (UniqueName: \"kubernetes.io/projected/6e7c563a-d986-46de-994c-230d665e028b-kube-api-access-6cmf9\") pod \"node-ca-8w4fn\" (UID: \"6e7c563a-d986-46de-994c-230d665e028b\") " pod="openshift-image-registry/node-ca-8w4fn"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200662    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8aa9591a-74d2-4890-be3f-482e458bd651-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200682    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-run-multus-certs\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200697    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-run-openvswitch\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.200762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200716    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-registration-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200851    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-host\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200891    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e7c563a-d986-46de-994c-230d665e028b-host\") pod \"node-ca-8w4fn\" (UID: \"6e7c563a-d986-46de-994c-230d665e028b\") " pod="openshift-image-registry/node-ca-8w4fn"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200920    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-hostroot\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200940    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-node-log\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200948    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6e7c563a-d986-46de-994c-230d665e028b-host\") pod \"node-ca-8w4fn\" (UID: \"6e7c563a-d986-46de-994c-230d665e028b\") " pod="openshift-image-registry/node-ca-8w4fn"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200958    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-log-socket\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.200983    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-cni-netd\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201006    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-sysctl-conf\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201024    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-run-ovn\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201043    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aa9591a-74d2-4890-be3f-482e458bd651-cni-binary-copy\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201062    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-etc-kubernetes\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201087    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cbeea170-d687-476c-b932-01800b00ecae-ovnkube-config\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201137    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-sysconfig\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201240    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-sysctl-d\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201279    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-system-cni-dir\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201308    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g62zk\" (UniqueName: \"kubernetes.io/projected/8aa9591a-74d2-4890-be3f-482e458bd651-kube-api-access-g62zk\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.201568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201335    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/98184b4a-870b-47be-a682-e1f7d43effb0-hosts-file\") pod \"node-resolver-4gmwh\" (UID: \"98184b4a-870b-47be-a682-e1f7d43effb0\") " pod="openshift-dns/node-resolver-4gmwh"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201341    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-system-cni-dir\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201360    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62299fb7-a41d-4628-9aa7-05a159ebbe07-cni-binary-copy\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201388    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-os-release\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201414    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-cnibin\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201448    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-os-release\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201474    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-run-k8s-cni-cncf-io\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201515    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-os-release\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201520    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-var-lib-kubelet\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201557    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-etc-openvswitch\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201574    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cbeea170-d687-476c-b932-01800b00ecae-ovn-node-metrics-cert\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201595    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-cnibin\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201621    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201647    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-cnibin\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201656    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98184b4a-870b-47be-a682-e1f7d43effb0-tmp-dir\") pod \"node-resolver-4gmwh\" (UID: \"98184b4a-870b-47be-a682-e1f7d43effb0\") " pod="openshift-dns/node-resolver-4gmwh"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201686    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-cni-dir\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201705    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-var-lib-cni-multus\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.202318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201721    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-var-lib-openvswitch\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201737    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cbeea170-d687-476c-b932-01800b00ecae-env-overrides\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201767    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-kubernetes\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201790    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-var-lib-cni-bin\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201827    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-conf-dir\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201842    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cbeea170-d687-476c-b932-01800b00ecae-ovnkube-script-lib\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201856    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-lib-modules\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201877    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dc8c62b-0158-46c7-8972-acf60187a450-tmp\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201894    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6e7c563a-d986-46de-994c-230d665e028b-serviceca\") pod \"node-ca-8w4fn\" (UID: \"6e7c563a-d986-46de-994c-230d665e028b\") " pod="openshift-image-registry/node-ca-8w4fn"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201930    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5-iptables-alerter-script\") pod \"iptables-alerter-8zxrq\" (UID: \"1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5\") " pod="openshift-network-operator/iptables-alerter-8zxrq"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201969    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5-host-slash\") pod \"iptables-alerter-8zxrq\" (UID: \"1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5\") " pod="openshift-network-operator/iptables-alerter-8zxrq"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.201996    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-run-netns\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.202032    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-device-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.202059    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-var-lib-kubelet\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.202087    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7ae93db-a16c-4e3b-bb52-59ccc91e033a-agent-certs\") pod \"konnectivity-agent-p7ftq\" (UID: \"b7ae93db-a16c-4e3b-bb52-59ccc91e033a\") " pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:25.202853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.202114    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hp67\" (UniqueName: \"kubernetes.io/projected/62299fb7-a41d-4628-9aa7-05a159ebbe07-kube-api-access-5hp67\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.203333 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.203239    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8aa9591a-74d2-4890-be3f-482e458bd651-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.203369 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.203339    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8aa9591a-74d2-4890-be3f-482e458bd651-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.203791 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.203776    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6e7c563a-d986-46de-994c-230d665e028b-serviceca\") pod \"node-ca-8w4fn\" (UID: \"6e7c563a-d986-46de-994c-230d665e028b\") " pod="openshift-image-registry/node-ca-8w4fn"
Jul 10 02:07:25.204641 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.204624    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8aa9591a-74d2-4890-be3f-482e458bd651-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.205322 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.205299    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8aa9591a-74d2-4890-be3f-482e458bd651-cni-binary-copy\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.208173 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.208153    2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 10 02:07:25.212028 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.212006    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cmf9\" (UniqueName: \"kubernetes.io/projected/6e7c563a-d986-46de-994c-230d665e028b-kube-api-access-6cmf9\") pod \"node-ca-8w4fn\" (UID: \"6e7c563a-d986-46de-994c-230d665e028b\") " pod="openshift-image-registry/node-ca-8w4fn"
Jul 10 02:07:25.212028 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.212006    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g62zk\" (UniqueName: \"kubernetes.io/projected/8aa9591a-74d2-4890-be3f-482e458bd651-kube-api-access-g62zk\") pod \"multus-additional-cni-plugins-7z79d\" (UID: \"8aa9591a-74d2-4890-be3f-482e458bd651\") " pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.218771 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.218755    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-24kwp"
Jul 10 02:07:25.230169 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.230152    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-24kwp"
Jul 10 02:07:25.302603 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302573    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-socket-dir-parent\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302618    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-modprobe-d\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302650    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-kubelet-config\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302675    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trztf\" (UniqueName: \"kubernetes.io/projected/98184b4a-870b-47be-a682-e1f7d43effb0-kube-api-access-trztf\") pod \"node-resolver-4gmwh\" (UID: \"98184b4a-870b-47be-a682-e1f7d43effb0\") " pod="openshift-dns/node-resolver-4gmwh"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302696    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-system-cni-dir\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302715    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-slash\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302739    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-run-multus-certs\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302764    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-run-openvswitch\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302770    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-modprobe-d\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302786    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-registration-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302776    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-socket-dir-parent\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302848    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-host\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302863    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-registration-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302880    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-hostroot\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302885    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-slash\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.302893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302889    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-system-cni-dir\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302921    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-run-openvswitch\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302937    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-node-log\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302965    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-hostroot\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302965    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-log-socket\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303009    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-node-log\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302969    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-run-multus-certs\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303009    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-host\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.302938    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-kubelet-config\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303032    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-cni-netd\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303065    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-sysctl-conf\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303073    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-cni-netd\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303091    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-run-ovn\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303117    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-etc-kubernetes\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303067    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-log-socket\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303152    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cbeea170-d687-476c-b932-01800b00ecae-ovnkube-config\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303168    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-run-ovn\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303182    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-sysconfig\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.303688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303189    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-sysctl-conf\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303206    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-sysctl-d\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303233    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/98184b4a-870b-47be-a682-e1f7d43effb0-hosts-file\") pod \"node-resolver-4gmwh\" (UID: \"98184b4a-870b-47be-a682-e1f7d43effb0\") " pod="openshift-dns/node-resolver-4gmwh"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303262    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62299fb7-a41d-4628-9aa7-05a159ebbe07-cni-binary-copy\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303264    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-sysconfig\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303285    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-cnibin\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303305    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/98184b4a-870b-47be-a682-e1f7d43effb0-hosts-file\") pod \"node-resolver-4gmwh\" (UID: \"98184b4a-870b-47be-a682-e1f7d43effb0\") " pod="openshift-dns/node-resolver-4gmwh"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303207    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-etc-kubernetes\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303313    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-sysctl-d\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303312    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-os-release\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303366    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-run-k8s-cni-cncf-io\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303371    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-os-release\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303372    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-cnibin\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303395    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-var-lib-kubelet\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303417    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-run-k8s-cni-cncf-io\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303423    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-etc-openvswitch\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303451    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cbeea170-d687-476c-b932-01800b00ecae-ovn-node-metrics-cert\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303460    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-var-lib-kubelet\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.304624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303480    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98184b4a-870b-47be-a682-e1f7d43effb0-tmp-dir\") pod \"node-resolver-4gmwh\" (UID: \"98184b4a-870b-47be-a682-e1f7d43effb0\") " pod="openshift-dns/node-resolver-4gmwh"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303510    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-cni-dir\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303535    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-etc-openvswitch\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303538    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-var-lib-cni-multus\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303569    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-var-lib-cni-multus\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303582    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-var-lib-openvswitch\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303610    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cbeea170-d687-476c-b932-01800b00ecae-env-overrides\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303635    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-kubernetes\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303659    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-var-lib-cni-bin\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303689    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-cni-dir\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303699    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-conf-dir\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303723    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cbeea170-d687-476c-b932-01800b00ecae-ovnkube-script-lib\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303749    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-lib-modules\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303774    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dc8c62b-0158-46c7-8972-acf60187a450-tmp\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303791    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/98184b4a-870b-47be-a682-e1f7d43effb0-tmp-dir\") pod \"node-resolver-4gmwh\" (UID: \"98184b4a-870b-47be-a682-e1f7d43effb0\") " pod="openshift-dns/node-resolver-4gmwh"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303799    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5-iptables-alerter-script\") pod \"iptables-alerter-8zxrq\" (UID: \"1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5\") " pod="openshift-network-operator/iptables-alerter-8zxrq"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303822    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5-host-slash\") pod \"iptables-alerter-8zxrq\" (UID: \"1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5\") " pod="openshift-network-operator/iptables-alerter-8zxrq"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303844    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cbeea170-d687-476c-b932-01800b00ecae-ovnkube-config\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.305624 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303866    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-run-netns\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303887    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-var-lib-cni-bin\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303895    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-device-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303913    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-var-lib-openvswitch\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303917    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-var-lib-kubelet\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303962    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7ae93db-a16c-4e3b-bb52-59ccc91e033a-agent-certs\") pod \"konnectivity-agent-p7ftq\" (UID: \"b7ae93db-a16c-4e3b-bb52-59ccc91e033a\") " pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303966    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/62299fb7-a41d-4628-9aa7-05a159ebbe07-cni-binary-copy\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303976    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-var-lib-kubelet\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.303987    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hp67\" (UniqueName: \"kubernetes.io/projected/62299fb7-a41d-4628-9aa7-05a159ebbe07-kube-api-access-5hp67\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304010    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-conf-dir\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304070    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304015    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304135    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-lib-modules\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304179    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-dbus\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304215    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-device-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304235    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-dbus\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304261    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-kubernetes\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.306464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304265    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304281    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/62299fb7-a41d-4628-9aa7-05a159ebbe07-host-run-netns\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304313    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cbeea170-d687-476c-b932-01800b00ecae-env-overrides\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304315    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsqgn\" (UniqueName: \"kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn\") pod \"network-check-target-st5kh\" (UID: \"a85578ca-e7d0-4036-b2c0-8315bf972f66\") " pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304326    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5-host-slash\") pod \"iptables-alerter-8zxrq\" (UID: \"1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5\") " pod="openshift-network-operator/iptables-alerter-8zxrq"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304354    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-daemon-config\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304400    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304519    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-sys-fs\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304550    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-systemd\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304562    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5-iptables-alerter-script\") pod \"iptables-alerter-8zxrq\" (UID: \"1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5\") " pod="openshift-network-operator/iptables-alerter-8zxrq"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304579    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-kubelet-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.304613    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304628    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-run\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304657    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9dc8c62b-0158-46c7-8972-acf60187a450-etc-tuned\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.304695    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret podName:796726ff-5fe1-48dd-9ea0-e5fae11d44e2 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:25.804667241 +0000 UTC m=+2.120977110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret") pod "global-pull-secret-syncer-mb7lt" (UID: "796726ff-5fe1-48dd-9ea0-e5fae11d44e2") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304839    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-run\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304892    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-sys-fs\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.307409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304935    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304938    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-etc-systemd\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304941    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cbeea170-d687-476c-b932-01800b00ecae-ovnkube-script-lib\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.304979    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xvfh\" (UniqueName: \"kubernetes.io/projected/1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5-kube-api-access-7xvfh\") pod \"iptables-alerter-8zxrq\" (UID: \"1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5\") " pod="openshift-network-operator/iptables-alerter-8zxrq"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305011    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-kubelet\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305036    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-run-netns\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305062    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-run-systemd\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.305071    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305087    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-run-ovn-kubernetes\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305118    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8pqn\" (UniqueName: \"kubernetes.io/projected/cbeea170-d687-476c-b932-01800b00ecae-kube-api-access-t8pqn\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.305138    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs podName:1a790021-08d0-4e49-8da8-f046861c72ac nodeName:}" failed. No retries permitted until 2026-07-10 02:07:25.805120441 +0000 UTC m=+2.121430307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs") pod "network-metrics-daemon-5j8nl" (UID: "1a790021-08d0-4e49-8da8-f046861c72ac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305157    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/62299fb7-a41d-4628-9aa7-05a159ebbe07-multus-daemon-config\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305177    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-sys\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305207    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-systemd-units\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305230    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-run-netns\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305235    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-cni-bin\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305244    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-run-systemd\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.308379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305271    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-socket-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305275    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-cni-bin\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305297    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-run-ovn-kubernetes\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305326    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9dc8c62b-0158-46c7-8972-acf60187a450-sys\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305326    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-host-kubelet\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305344    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cbeea170-d687-476c-b932-01800b00ecae-systemd-units\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305349    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2lxt\" (UniqueName: \"kubernetes.io/projected/c541937e-9bfd-48fe-a99f-c97f0e4780f6-kube-api-access-p2lxt\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305378    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305406    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gbg5\" (UniqueName: \"kubernetes.io/projected/9dc8c62b-0158-46c7-8972-acf60187a450-kube-api-access-6gbg5\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305421    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-socket-dir\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305434    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddrk4\" (UniqueName: \"kubernetes.io/projected/1a790021-08d0-4e49-8da8-f046861c72ac-kube-api-access-ddrk4\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305523    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c541937e-9bfd-48fe-a99f-c97f0e4780f6-etc-selinux\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.305559    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7ae93db-a16c-4e3b-bb52-59ccc91e033a-konnectivity-ca\") pod \"konnectivity-agent-p7ftq\" (UID: \"b7ae93db-a16c-4e3b-bb52-59ccc91e033a\") " pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.306221    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7ae93db-a16c-4e3b-bb52-59ccc91e033a-konnectivity-ca\") pod \"konnectivity-agent-p7ftq\" (UID: \"b7ae93db-a16c-4e3b-bb52-59ccc91e033a\") " pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.306519    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cbeea170-d687-476c-b932-01800b00ecae-ovn-node-metrics-cert\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.306958    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9dc8c62b-0158-46c7-8972-acf60187a450-etc-tuned\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.307435    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9dc8c62b-0158-46c7-8972-acf60187a450-tmp\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.309264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.308065    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7ae93db-a16c-4e3b-bb52-59ccc91e033a-agent-certs\") pod \"konnectivity-agent-p7ftq\" (UID: \"b7ae93db-a16c-4e3b-bb52-59ccc91e033a\") " pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:25.315335 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.315316    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:07:25.315335 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.315338    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:07:25.315472 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.315349    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fsqgn for pod openshift-network-diagnostics/network-check-target-st5kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:25.315472 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.315408    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn podName:a85578ca-e7d0-4036-b2c0-8315bf972f66 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:25.81538826 +0000 UTC m=+2.131698154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fsqgn" (UniqueName: "kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn") pod "network-check-target-st5kh" (UID: "a85578ca-e7d0-4036-b2c0-8315bf972f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:25.318066 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.318002    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trztf\" (UniqueName: \"kubernetes.io/projected/98184b4a-870b-47be-a682-e1f7d43effb0-kube-api-access-trztf\") pod \"node-resolver-4gmwh\" (UID: \"98184b4a-870b-47be-a682-e1f7d43effb0\") " pod="openshift-dns/node-resolver-4gmwh"
Jul 10 02:07:25.320135 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.320107    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddrk4\" (UniqueName: \"kubernetes.io/projected/1a790021-08d0-4e49-8da8-f046861c72ac-kube-api-access-ddrk4\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:25.320343 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.320323    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8pqn\" (UniqueName: \"kubernetes.io/projected/cbeea170-d687-476c-b932-01800b00ecae-kube-api-access-t8pqn\") pod \"ovnkube-node-r4vzs\" (UID: \"cbeea170-d687-476c-b932-01800b00ecae\") " pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.320417 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.320330    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hp67\" (UniqueName: \"kubernetes.io/projected/62299fb7-a41d-4628-9aa7-05a159ebbe07-kube-api-access-5hp67\") pod \"multus-qjjm5\" (UID: \"62299fb7-a41d-4628-9aa7-05a159ebbe07\") " pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.320473 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.320441    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2lxt\" (UniqueName: \"kubernetes.io/projected/c541937e-9bfd-48fe-a99f-c97f0e4780f6-kube-api-access-p2lxt\") pod \"aws-ebs-csi-driver-node-dldnw\" (UID: \"c541937e-9bfd-48fe-a99f-c97f0e4780f6\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.320855 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.320832    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gbg5\" (UniqueName: \"kubernetes.io/projected/9dc8c62b-0158-46c7-8972-acf60187a450-kube-api-access-6gbg5\") pod \"tuned-jmm5j\" (UID: \"9dc8c62b-0158-46c7-8972-acf60187a450\") " pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.320977 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.320861    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xvfh\" (UniqueName: \"kubernetes.io/projected/1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5-kube-api-access-7xvfh\") pod \"iptables-alerter-8zxrq\" (UID: \"1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5\") " pod="openshift-network-operator/iptables-alerter-8zxrq"
Jul 10 02:07:25.418260 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:25.418221    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b060fe845b58018afc31fc696c483a8.slice/crio-a7c654e088edecb3ccb9425d95766f664ad7a0acb91541c20d7831caf364bda5 WatchSource:0}: Error finding container a7c654e088edecb3ccb9425d95766f664ad7a0acb91541c20d7831caf364bda5: Status 404 returned error can't find the container with id a7c654e088edecb3ccb9425d95766f664ad7a0acb91541c20d7831caf364bda5
Jul 10 02:07:25.425597 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.425570    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:07:25.430531 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.430510    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8w4fn"
Jul 10 02:07:25.430742 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:25.430723    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0d70d9ea477f5082aeebc1d283121de.slice/crio-75d3efb67acbba7f6e2273b7e480df40e29165440e3e75c2eff100cf906df706 WatchSource:0}: Error finding container 75d3efb67acbba7f6e2273b7e480df40e29165440e3e75c2eff100cf906df706: Status 404 returned error can't find the container with id 75d3efb67acbba7f6e2273b7e480df40e29165440e3e75c2eff100cf906df706
Jul 10 02:07:25.439285 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:25.439258    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e7c563a_d986_46de_994c_230d665e028b.slice/crio-c849d06ce0470815d1c0f74372262e1552b1761c0a6509a4096524fd8c316978 WatchSource:0}: Error finding container c849d06ce0470815d1c0f74372262e1552b1761c0a6509a4096524fd8c316978: Status 404 returned error can't find the container with id c849d06ce0470815d1c0f74372262e1552b1761c0a6509a4096524fd8c316978
Jul 10 02:07:25.450768 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.450744    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7z79d"
Jul 10 02:07:25.459198 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:25.459174    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa9591a_74d2_4890_be3f_482e458bd651.slice/crio-feaf2ffa1aa78c53bf28db0f4cff0aeb7d953eda5c46c0d015e03eaf31f595e8 WatchSource:0}: Error finding container feaf2ffa1aa78c53bf28db0f4cff0aeb7d953eda5c46c0d015e03eaf31f595e8: Status 404 returned error can't find the container with id feaf2ffa1aa78c53bf28db0f4cff0aeb7d953eda5c46c0d015e03eaf31f595e8
Jul 10 02:07:25.473955 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.473936    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8zxrq"
Jul 10 02:07:25.480020 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.479998    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jmm5j"
Jul 10 02:07:25.482680 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:25.482654    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d597e84_1be6_4f55_a9ef_6f3f4e2f3db5.slice/crio-374a75947789b5284c369fc9de778ff961d052a79465d429688e048289077a35 WatchSource:0}: Error finding container 374a75947789b5284c369fc9de778ff961d052a79465d429688e048289077a35: Status 404 returned error can't find the container with id 374a75947789b5284c369fc9de778ff961d052a79465d429688e048289077a35
Jul 10 02:07:25.488607 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:25.488582    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dc8c62b_0158_46c7_8972_acf60187a450.slice/crio-b1d7e00abd08a23b7c9be4259525ce89f46926de47db0d3ae56d35e2772c9b6c WatchSource:0}: Error finding container b1d7e00abd08a23b7c9be4259525ce89f46926de47db0d3ae56d35e2772c9b6c: Status 404 returned error can't find the container with id b1d7e00abd08a23b7c9be4259525ce89f46926de47db0d3ae56d35e2772c9b6c
Jul 10 02:07:25.506908 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.506886    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4gmwh"
Jul 10 02:07:25.517712 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:25.517687    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98184b4a_870b_47be_a682_e1f7d43effb0.slice/crio-9cb4d8ab0374e55e33ff8301eb098f1cfee1bb59d0db85439a76e7bad30e0b25 WatchSource:0}: Error finding container 9cb4d8ab0374e55e33ff8301eb098f1cfee1bb59d0db85439a76e7bad30e0b25: Status 404 returned error can't find the container with id 9cb4d8ab0374e55e33ff8301eb098f1cfee1bb59d0db85439a76e7bad30e0b25
Jul 10 02:07:25.526589 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.526571    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qjjm5"
Jul 10 02:07:25.534514 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:25.534491    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62299fb7_a41d_4628_9aa7_05a159ebbe07.slice/crio-3fd5f36d55f130e935aa637c47c62a0d79ad43e5a5a7d4a9da86e00ba7f012db WatchSource:0}: Error finding container 3fd5f36d55f130e935aa637c47c62a0d79ad43e5a5a7d4a9da86e00ba7f012db: Status 404 returned error can't find the container with id 3fd5f36d55f130e935aa637c47c62a0d79ad43e5a5a7d4a9da86e00ba7f012db
Jul 10 02:07:25.537766 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.537747    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:25.545731 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:25.545710    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbeea170_d687_476c_b932_01800b00ecae.slice/crio-bec58795c22e1d4c071650e17da0b682cc6a4f233b94b1ac2459a3fd154f1d5d WatchSource:0}: Error finding container bec58795c22e1d4c071650e17da0b682cc6a4f233b94b1ac2459a3fd154f1d5d: Status 404 returned error can't find the container with id bec58795c22e1d4c071650e17da0b682cc6a4f233b94b1ac2459a3fd154f1d5d
Jul 10 02:07:25.551710 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.551693    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:25.552296 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.552280    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:07:25.561321 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:25.561296    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ae93db_a16c_4e3b_bb52_59ccc91e033a.slice/crio-10219cc0611e1253de067e58b7f4dc138ff30aec974bd06963d20aa30a43094d WatchSource:0}: Error finding container 10219cc0611e1253de067e58b7f4dc138ff30aec974bd06963d20aa30a43094d: Status 404 returned error can't find the container with id 10219cc0611e1253de067e58b7f4dc138ff30aec974bd06963d20aa30a43094d
Jul 10 02:07:25.561321 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.561305    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw"
Jul 10 02:07:25.569505 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:07:25.569484    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc541937e_9bfd_48fe_a99f_c97f0e4780f6.slice/crio-1ecd956fc38f3e224067ea97d6d4d5098a27a5235db020b2e9e08c18e863b39a WatchSource:0}: Error finding container 1ecd956fc38f3e224067ea97d6d4d5098a27a5235db020b2e9e08c18e863b39a: Status 404 returned error can't find the container with id 1ecd956fc38f3e224067ea97d6d4d5098a27a5235db020b2e9e08c18e863b39a
Jul 10 02:07:25.571119 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.571100    2557 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-driver start failed in pod aws-ebs-csi-driver-node-dldnw_openshift-cluster-csi-drivers(c541937e-9bfd-48fe-a99f-c97f0e4780f6): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 10 02:07:25.571871 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.571840    2557 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-node-driver-registrar start failed in pod aws-ebs-csi-driver-node-dldnw_openshift-cluster-csi-drivers(c541937e-9bfd-48fe-a99f-c97f0e4780f6): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 10 02:07:25.573670 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.573653    2557 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-liveness-probe start failed in pod aws-ebs-csi-driver-node-dldnw_openshift-cluster-csi-drivers(c541937e-9bfd-48fe-a99f-c97f0e4780f6): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Jul 10 02:07:25.573714 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.573692    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw" podUID="c541937e-9bfd-48fe-a99f-c97f0e4780f6"
Jul 10 02:07:25.809581 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.809488    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:25.809581 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.809537    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:25.809798 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.809619    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:25.809798 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.809659    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:25.809798 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.809696    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret podName:796726ff-5fe1-48dd-9ea0-e5fae11d44e2 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:26.809672402 +0000 UTC m=+3.125982280 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret") pod "global-pull-secret-syncer-mb7lt" (UID: "796726ff-5fe1-48dd-9ea0-e5fae11d44e2") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:25.809798 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.809714    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs podName:1a790021-08d0-4e49-8da8-f046861c72ac nodeName:}" failed. No retries permitted until 2026-07-10 02:07:26.809706557 +0000 UTC m=+3.126016408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs") pod "network-metrics-daemon-5j8nl" (UID: "1a790021-08d0-4e49-8da8-f046861c72ac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:25.910070 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:25.910027    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsqgn\" (UniqueName: \"kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn\") pod \"network-check-target-st5kh\" (UID: \"a85578ca-e7d0-4036-b2c0-8315bf972f66\") " pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:25.910264 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.910219    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:07:25.910264 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.910246    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:07:25.910264 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.910259    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fsqgn for pod openshift-network-diagnostics/network-check-target-st5kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:25.910430 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:25.910335    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn podName:a85578ca-e7d0-4036-b2c0-8315bf972f66 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:26.910313255 +0000 UTC m=+3.226623129 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fsqgn" (UniqueName: "kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn") pod "network-check-target-st5kh" (UID: "a85578ca-e7d0-4036-b2c0-8315bf972f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:26.013978 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.013937    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:07:26.244651 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.234638    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:02:26 +0000 UTC" deadline="2028-03-17 13:09:25.216987962 +0000 UTC"
Jul 10 02:07:26.244651 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.234697    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14795h1m58.982294537s"
Jul 10 02:07:26.244651 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.238650    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:26.244651 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:26.238776    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:26.244651 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.239345    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:26.244651 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:26.239461    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:26.246159 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.245998    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p7ftq" event={"ID":"b7ae93db-a16c-4e3b-bb52-59ccc91e033a","Type":"ContainerStarted","Data":"10219cc0611e1253de067e58b7f4dc138ff30aec974bd06963d20aa30a43094d"}
Jul 10 02:07:26.248313 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.248229    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jmm5j" event={"ID":"9dc8c62b-0158-46c7-8972-acf60187a450","Type":"ContainerStarted","Data":"b1d7e00abd08a23b7c9be4259525ce89f46926de47db0d3ae56d35e2772c9b6c"}
Jul 10 02:07:26.249822 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.249747    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z79d" event={"ID":"8aa9591a-74d2-4890-be3f-482e458bd651","Type":"ContainerStarted","Data":"feaf2ffa1aa78c53bf28db0f4cff0aeb7d953eda5c46c0d015e03eaf31f595e8"}
Jul 10 02:07:26.251801 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.251732    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal" event={"ID":"3b060fe845b58018afc31fc696c483a8","Type":"ContainerStarted","Data":"a7c654e088edecb3ccb9425d95766f664ad7a0acb91541c20d7831caf364bda5"}
Jul 10 02:07:26.257166 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.256231    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw" event={"ID":"c541937e-9bfd-48fe-a99f-c97f0e4780f6","Type":"ContainerStarted","Data":"1ecd956fc38f3e224067ea97d6d4d5098a27a5235db020b2e9e08c18e863b39a"}
Jul 10 02:07:26.259509 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.259483    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs" event={"ID":"cbeea170-d687-476c-b932-01800b00ecae","Type":"ContainerStarted","Data":"bec58795c22e1d4c071650e17da0b682cc6a4f233b94b1ac2459a3fd154f1d5d"}
Jul 10 02:07:26.265294 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:26.265248    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:23e38a7cf7c4e5c572a5461047f39d2f821e55490088f442a842f66da801ae99\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0512f4cb34792d3c7553c42a85869c34b5953550c019b958649e3246d6540d94\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0f984e9f25655116e61604ac13881b82e2830789119e02b44a338cea95f30425\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw" podUID="c541937e-9bfd-48fe-a99f-c97f0e4780f6"
Jul 10 02:07:26.267291 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.267265    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qjjm5" event={"ID":"62299fb7-a41d-4628-9aa7-05a159ebbe07","Type":"ContainerStarted","Data":"3fd5f36d55f130e935aa637c47c62a0d79ad43e5a5a7d4a9da86e00ba7f012db"}
Jul 10 02:07:26.271239 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.271211    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4gmwh" event={"ID":"98184b4a-870b-47be-a682-e1f7d43effb0","Type":"ContainerStarted","Data":"9cb4d8ab0374e55e33ff8301eb098f1cfee1bb59d0db85439a76e7bad30e0b25"}
Jul 10 02:07:26.274339 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.274254    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8zxrq" event={"ID":"1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5","Type":"ContainerStarted","Data":"374a75947789b5284c369fc9de778ff961d052a79465d429688e048289077a35"}
Jul 10 02:07:26.278797 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.278759    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8w4fn" event={"ID":"6e7c563a-d986-46de-994c-230d665e028b","Type":"ContainerStarted","Data":"c849d06ce0470815d1c0f74372262e1552b1761c0a6509a4096524fd8c316978"}
Jul 10 02:07:26.281136 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.281081    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-213.ec2.internal" event={"ID":"d0d70d9ea477f5082aeebc1d283121de","Type":"ContainerStarted","Data":"75d3efb67acbba7f6e2273b7e480df40e29165440e3e75c2eff100cf906df706"}
Jul 10 02:07:26.819081 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.819038    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:26.819376 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.819111    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:26.819376 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:26.819277    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:26.819376 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:26.819348    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs podName:1a790021-08d0-4e49-8da8-f046861c72ac nodeName:}" failed. No retries permitted until 2026-07-10 02:07:28.819328019 +0000 UTC m=+5.135637878 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs") pod "network-metrics-daemon-5j8nl" (UID: "1a790021-08d0-4e49-8da8-f046861c72ac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:26.819809 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:26.819788    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:26.819967 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:26.819852    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret podName:796726ff-5fe1-48dd-9ea0-e5fae11d44e2 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:28.819834293 +0000 UTC m=+5.136144155 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret") pod "global-pull-secret-syncer-mb7lt" (UID: "796726ff-5fe1-48dd-9ea0-e5fae11d44e2") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:26.919650 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:26.919606    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsqgn\" (UniqueName: \"kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn\") pod \"network-check-target-st5kh\" (UID: \"a85578ca-e7d0-4036-b2c0-8315bf972f66\") " pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:26.919881 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:26.919850    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:07:26.919881 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:26.919878    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:07:26.920009 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:26.919893    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fsqgn for pod openshift-network-diagnostics/network-check-target-st5kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:26.920009 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:26.919961    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn podName:a85578ca-e7d0-4036-b2c0-8315bf972f66 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:28.919940773 +0000 UTC m=+5.236250629 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fsqgn" (UniqueName: "kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn") pod "network-check-target-st5kh" (UID: "a85578ca-e7d0-4036-b2c0-8315bf972f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:27.236018 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:27.235941    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:27.236466 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:27.236087    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:27.236530 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:27.236455    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 02:02:26 +0000 UTC" deadline="2028-01-10 17:29:27.898148267 +0000 UTC"
Jul 10 02:07:27.236530 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:27.236479    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13191h22m0.661673855s"
Jul 10 02:07:27.954290 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:27.954066    2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 10 02:07:28.239112 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:28.239033    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:28.239564 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:28.239162    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:28.239564 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:28.239557    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:28.239684 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:28.239669    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:28.835316 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:28.835226    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:28.836052 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:28.835411    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:28.836052 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:28.835638    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret podName:796726ff-5fe1-48dd-9ea0-e5fae11d44e2 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:32.835613241 +0000 UTC m=+9.151923117 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret") pod "global-pull-secret-syncer-mb7lt" (UID: "796726ff-5fe1-48dd-9ea0-e5fae11d44e2") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:28.836333 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:28.836290    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:28.836725 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:28.836553    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:28.836725 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:28.836703    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs podName:1a790021-08d0-4e49-8da8-f046861c72ac nodeName:}" failed. No retries permitted until 2026-07-10 02:07:32.836683225 +0000 UTC m=+9.152993081 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs") pod "network-metrics-daemon-5j8nl" (UID: "1a790021-08d0-4e49-8da8-f046861c72ac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:28.938955 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:28.938254    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsqgn\" (UniqueName: \"kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn\") pod \"network-check-target-st5kh\" (UID: \"a85578ca-e7d0-4036-b2c0-8315bf972f66\") " pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:28.938955 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:28.938488    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:07:28.938955 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:28.938508    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:07:28.938955 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:28.938521    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fsqgn for pod openshift-network-diagnostics/network-check-target-st5kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:28.938955 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:28.938579    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn podName:a85578ca-e7d0-4036-b2c0-8315bf972f66 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:32.938560526 +0000 UTC m=+9.254870383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fsqgn" (UniqueName: "kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn") pod "network-check-target-st5kh" (UID: "a85578ca-e7d0-4036-b2c0-8315bf972f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:29.236641 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:29.236552    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:29.236808 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:29.236692    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:30.236684 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:30.236647    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:30.237186 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:30.236813    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:30.237186 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:30.236875    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:30.237186 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:30.236955    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:31.235800 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:31.235755    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:31.236011 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:31.235921    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:32.236097 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:32.236057    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:32.236097 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:32.236093    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:32.236659 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:32.236224    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:32.236659 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:32.236344    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:32.874935 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:32.874876    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:32.875136 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:32.874960    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret podName:796726ff-5fe1-48dd-9ea0-e5fae11d44e2 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:40.874940876 +0000 UTC m=+17.191250732 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret") pod "global-pull-secret-syncer-mb7lt" (UID: "796726ff-5fe1-48dd-9ea0-e5fae11d44e2") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:32.875596 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:32.874873    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:32.875596 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:32.875404    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:32.875596 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:32.875573    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:32.875865 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:32.875636    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs podName:1a790021-08d0-4e49-8da8-f046861c72ac nodeName:}" failed. No retries permitted until 2026-07-10 02:07:40.875619581 +0000 UTC m=+17.191929448 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs") pod "network-metrics-daemon-5j8nl" (UID: "1a790021-08d0-4e49-8da8-f046861c72ac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:32.976568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:32.976465    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsqgn\" (UniqueName: \"kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn\") pod \"network-check-target-st5kh\" (UID: \"a85578ca-e7d0-4036-b2c0-8315bf972f66\") " pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:32.976732 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:32.976685    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:07:32.976732 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:32.976715    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:07:32.976732 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:32.976730    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fsqgn for pod openshift-network-diagnostics/network-check-target-st5kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:32.976926 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:32.976800    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn podName:a85578ca-e7d0-4036-b2c0-8315bf972f66 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:40.976779305 +0000 UTC m=+17.293089161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fsqgn" (UniqueName: "kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn") pod "network-check-target-st5kh" (UID: "a85578ca-e7d0-4036-b2c0-8315bf972f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:33.235720 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:33.235628    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:33.235919 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:33.235773    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:34.237462 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:34.237431    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:34.237993 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:34.237556    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:34.237993 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:34.237626    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:34.237993 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:34.237701    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:35.236128 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:35.236093    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:35.236309 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:35.236242    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:36.235980 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:36.235952    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:36.236415 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:36.236081    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:36.236415 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:36.236150    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:36.236415 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:36.236252    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:37.236481 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:37.236308    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:37.236943 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:37.236568    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:37.318357 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:37.318323    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jmm5j" event={"ID":"9dc8c62b-0158-46c7-8972-acf60187a450","Type":"ContainerStarted","Data":"50a2cd0a30a44fb73bdde64673ea6bba11aacf8efd0df3339ec3409f1996aa8d"}
Jul 10 02:07:37.320362 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:37.320315    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-213.ec2.internal" event={"ID":"d0d70d9ea477f5082aeebc1d283121de","Type":"ContainerStarted","Data":"ad3ea880d2fcc27c617fa4942c241eee6d282ca4bc1b861f410efd8d39354a9e"}
Jul 10 02:07:37.351385 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:37.351314    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-213.ec2.internal" podStartSLOduration=12.351293262 podStartE2EDuration="12.351293262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:07:37.351269584 +0000 UTC m=+13.667579509" watchObservedRunningTime="2026-07-10 02:07:37.351293262 +0000 UTC m=+13.667603170"
Jul 10 02:07:37.351961 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:37.351922    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jmm5j" podStartSLOduration=1.5277160570000001 podStartE2EDuration="12.351909585s" totalImagesPullingTime="10.824193528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:07:37.337699468 +0000 UTC m=+13.654009341" watchObservedRunningTime="2026-07-10 02:07:37.351909585 +0000 UTC m=+13.668219459"
Jul 10 02:07:38.235877 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.235851    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:38.235877 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.235876    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:38.236104 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:38.235972    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:38.236104 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:38.236031    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:38.323472 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.323431    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p7ftq" event={"ID":"b7ae93db-a16c-4e3b-bb52-59ccc91e033a","Type":"ContainerStarted","Data":"10e054da73861479ef845a1cddacac5bce68773b2bbf1da8b696e1196c2b03b5"}
Jul 10 02:07:38.325102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.325067    2557 generic.go:356] "Generic (PLEG): container finished" podID="8aa9591a-74d2-4890-be3f-482e458bd651" containerID="db532ec064e731e0ff7a757668036ba2114c5eb48f05b734763f1d9be0eb4936" exitCode=0
Jul 10 02:07:38.325210 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.325099    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z79d" event={"ID":"8aa9591a-74d2-4890-be3f-482e458bd651","Type":"ContainerDied","Data":"db532ec064e731e0ff7a757668036ba2114c5eb48f05b734763f1d9be0eb4936"}
Jul 10 02:07:38.326763 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.326732    2557 generic.go:356] "Generic (PLEG): container finished" podID="3b060fe845b58018afc31fc696c483a8" containerID="84f80b9b4211c280fb4203d8b8e331b586a8ae945e7ca4002f8b300b9c3a6ffe" exitCode=0
Jul 10 02:07:38.327213 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.327158    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal" event={"ID":"3b060fe845b58018afc31fc696c483a8","Type":"ContainerDied","Data":"84f80b9b4211c280fb4203d8b8e331b586a8ae945e7ca4002f8b300b9c3a6ffe"}
Jul 10 02:07:38.331561 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.331532    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4gmwh" event={"ID":"98184b4a-870b-47be-a682-e1f7d43effb0","Type":"ContainerStarted","Data":"33cf5161fa046eb3c22610c7fdfc71fe0518d00c3a008f06cde0aa25af0d1ecf"}
Jul 10 02:07:38.333060 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.333033    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8zxrq" event={"ID":"1d597e84-1be6-4f55-a9ef-6f3f4e2f3db5","Type":"ContainerStarted","Data":"4822a96d841e58e005d69ed4415f6d4cfc7a1cd54cf8f74c11000cc656244432"}
Jul 10 02:07:38.334526 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.334499    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8w4fn" event={"ID":"6e7c563a-d986-46de-994c-230d665e028b","Type":"ContainerStarted","Data":"abc26d848d7988c91d3a2a4f74a0e2c77a85b284fcef067cab4bae9dc6d62b1b"}
Jul 10 02:07:38.339156 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.339116    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p7ftq" podStartSLOduration=2.805938782 podStartE2EDuration="13.339100612s" totalImagesPullingTime="10.53316183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:07:38.338619412 +0000 UTC m=+14.654929285" watchObservedRunningTime="2026-07-10 02:07:38.339100612 +0000 UTC m=+14.655410491"
Jul 10 02:07:38.352686 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.352648    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4gmwh" podStartSLOduration=2.776832427 podStartE2EDuration="13.352635442s" totalImagesPullingTime="10.575803015s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:07:38.352529721 +0000 UTC m=+14.668839594" watchObservedRunningTime="2026-07-10 02:07:38.352635442 +0000 UTC m=+14.668945314"
Jul 10 02:07:38.367023 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.366982    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8w4fn" podStartSLOduration=2.712530019 podStartE2EDuration="13.366971496s" totalImagesPullingTime="10.654441477s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:07:38.366529787 +0000 UTC m=+14.682839660" watchObservedRunningTime="2026-07-10 02:07:38.366971496 +0000 UTC m=+14.683281371"
Jul 10 02:07:38.398099 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:38.398046    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8zxrq" podStartSLOduration=2.591643838 podStartE2EDuration="13.398029203s" totalImagesPullingTime="10.806385365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:07:38.381636458 +0000 UTC m=+14.697946354" watchObservedRunningTime="2026-07-10 02:07:38.398029203 +0000 UTC m=+14.714339077"
Jul 10 02:07:39.236234 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:39.236044    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:39.236445 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:39.236329    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:39.339967 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:39.339935    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal" event={"ID":"3b060fe845b58018afc31fc696c483a8","Type":"ContainerStarted","Data":"041bbaf4b0c776286feb4ae31e6f5940145e22ac3527b8c81c8257a06b3dfcac"}
Jul 10 02:07:39.356107 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:39.356048    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-213.ec2.internal" podStartSLOduration=14.356026732 podStartE2EDuration="14.356026732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:07:39.355259861 +0000 UTC m=+15.671569760" watchObservedRunningTime="2026-07-10 02:07:39.356026732 +0000 UTC m=+15.672336608"
Jul 10 02:07:40.239445 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:40.239416    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:40.239445 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:40.239439    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:40.239679 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:40.239538    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:40.239679 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:40.239654    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:40.937420 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:40.937380    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:40.938107 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:40.937431    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:40.938107 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:40.937545    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:40.938107 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:40.937563    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:40.938107 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:40.937615    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs podName:1a790021-08d0-4e49-8da8-f046861c72ac nodeName:}" failed. No retries permitted until 2026-07-10 02:07:56.937600694 +0000 UTC m=+33.253910544 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs") pod "network-metrics-daemon-5j8nl" (UID: "1a790021-08d0-4e49-8da8-f046861c72ac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:40.938107 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:40.937629    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret podName:796726ff-5fe1-48dd-9ea0-e5fae11d44e2 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:56.93762312 +0000 UTC m=+33.253932970 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret") pod "global-pull-secret-syncer-mb7lt" (UID: "796726ff-5fe1-48dd-9ea0-e5fae11d44e2") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:40.944164 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:40.944141    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:40.944757 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:40.944735    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:41.038594 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:41.038555    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsqgn\" (UniqueName: \"kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn\") pod \"network-check-target-st5kh\" (UID: \"a85578ca-e7d0-4036-b2c0-8315bf972f66\") " pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:41.038788 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:41.038765    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:07:41.038788 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:41.038788    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:07:41.038889 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:41.038816    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fsqgn for pod openshift-network-diagnostics/network-check-target-st5kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:41.038932 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:41.038899    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn podName:a85578ca-e7d0-4036-b2c0-8315bf972f66 nodeName:}" failed. No retries permitted until 2026-07-10 02:07:57.038879985 +0000 UTC m=+33.355189850 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fsqgn" (UniqueName: "kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn") pod "network-check-target-st5kh" (UID: "a85578ca-e7d0-4036-b2c0-8315bf972f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:41.236076 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:41.236005    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:41.236306 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:41.236143    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:42.236747 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:42.236715    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:42.237350 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:42.236863    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:42.237350 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:42.237206    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:42.237350 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:42.237312    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:43.235839 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:43.235794    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:43.236012 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:43.235915    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:44.236764 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:44.236704    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:44.237259 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:44.236832    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:44.237259 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:44.236867    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:44.237259 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:44.236971    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:44.951786 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:44.951745    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:44.951964 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:44.951909    2557 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 10 02:07:44.952396 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:44.952378    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p7ftq"
Jul 10 02:07:45.235943 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:45.235865    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:45.236094 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:45.235988    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:46.235691 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:46.235603    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:46.236262 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:46.235720    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:46.236262 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:46.235814    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:46.236262 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:46.235918    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:47.236467 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:47.236434    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:47.236925 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:47.236563    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:48.235957 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:48.235923    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:48.235957 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:48.235968    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:48.236280 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:48.236061    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:48.236280 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:48.236202    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:49.235610 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:49.235581    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:49.236066 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:49.235680    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:50.236152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:50.236117    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:50.236558 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:50.236270    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:50.236558 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:50.236340    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:50.236558 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:50.236456    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:51.236701 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.236392    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:51.237159 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:51.236869    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:51.364285 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.364248    2557 generic.go:356] "Generic (PLEG): container finished" podID="8aa9591a-74d2-4890-be3f-482e458bd651" containerID="e51b438012630c2d6868623f3cd9d9dd54bba1e81c488927f82821877f6beebd" exitCode=0
Jul 10 02:07:51.364437 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.364324    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z79d" event={"ID":"8aa9591a-74d2-4890-be3f-482e458bd651","Type":"ContainerDied","Data":"e51b438012630c2d6868623f3cd9d9dd54bba1e81c488927f82821877f6beebd"}
Jul 10 02:07:51.365914 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.365888    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw" event={"ID":"c541937e-9bfd-48fe-a99f-c97f0e4780f6","Type":"ContainerStarted","Data":"861de35734a402cfa31b7befbce8f83b294e7d0383985ec151c18749a368942e"}
Jul 10 02:07:51.368644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.368623    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:07:51.368997 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.368978    2557 generic.go:356] "Generic (PLEG): container finished" podID="cbeea170-d687-476c-b932-01800b00ecae" containerID="b95a907d9defe1f02c36813209db92b173818dd6763974e45db9376f83a75d68" exitCode=1
Jul 10 02:07:51.369072 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.369052    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs" event={"ID":"cbeea170-d687-476c-b932-01800b00ecae","Type":"ContainerStarted","Data":"0065542df161b33fd821b8252a933253e993b402e083d09eeeb6fe38c51cf102"}
Jul 10 02:07:51.369144 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.369132    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs" event={"ID":"cbeea170-d687-476c-b932-01800b00ecae","Type":"ContainerStarted","Data":"43c226730ed3a3f7d9de01ac113bff1efcda1a2c06e966ad5422560dfa696e51"}
Jul 10 02:07:51.369197 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.369184    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs" event={"ID":"cbeea170-d687-476c-b932-01800b00ecae","Type":"ContainerStarted","Data":"31ddee77b640a1312976aac90e4389b4f973d26c86eb31b7a3cddb6afae538eb"}
Jul 10 02:07:51.369248 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.369235    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs" event={"ID":"cbeea170-d687-476c-b932-01800b00ecae","Type":"ContainerStarted","Data":"61dfbfc1412659974f080428e156f154037dd03c5d2c1ff8f095b378e48fae84"}
Jul 10 02:07:51.369300 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.369285    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs" event={"ID":"cbeea170-d687-476c-b932-01800b00ecae","Type":"ContainerDied","Data":"b95a907d9defe1f02c36813209db92b173818dd6763974e45db9376f83a75d68"}
Jul 10 02:07:51.369350 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.369335    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs" event={"ID":"cbeea170-d687-476c-b932-01800b00ecae","Type":"ContainerStarted","Data":"55c5cdbbb6bc7633db8c4fa6372f2028d5e73e07633a09b6e5b784bd786cfca9"}
Jul 10 02:07:51.370330 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.370311    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qjjm5" event={"ID":"62299fb7-a41d-4628-9aa7-05a159ebbe07","Type":"ContainerStarted","Data":"57f06ab3878e6a4a5679f09e62d98b41a6b54f102837dbec9162c3639b479988"}
Jul 10 02:07:51.409486 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:51.409428    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qjjm5" podStartSLOduration=1.496778084 podStartE2EDuration="26.40940737s" totalImagesPullingTime="24.912629286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:07:51.408979158 +0000 UTC m=+27.725289030" watchObservedRunningTime="2026-07-10 02:07:51.40940737 +0000 UTC m=+27.725717243"
Jul 10 02:07:52.060393 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:52.060356    2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 10 02:07:52.174119 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:52.174022    2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T02:07:52.060377012Z","UUID":"062867ea-cecb-455f-b209-e207a9ff5d51","Handler":null,"Name":"","Endpoint":""}
Jul 10 02:07:52.175498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:52.175473    2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 10 02:07:52.175632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:52.175521    2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 10 02:07:52.235910 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:52.235829    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:52.236073 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:52.235829    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:52.236073 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:52.235959    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:52.236073 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:52.236063    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:52.377750 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:52.377720    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw" event={"ID":"c541937e-9bfd-48fe-a99f-c97f0e4780f6","Type":"ContainerStarted","Data":"168845d25b70308af4a29d2b4b63705c783d8b43a4bc67ee5ffc0166742b0bc5"}
Jul 10 02:07:53.235814 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:53.235764    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:53.236024 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:53.235908    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:53.381902 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:53.381866    2557 generic.go:356] "Generic (PLEG): container finished" podID="8aa9591a-74d2-4890-be3f-482e458bd651" containerID="78d1af60b5c8d314e0bf21138960ff3917db404674c7032692a2ceb17e0b5c4d" exitCode=0
Jul 10 02:07:53.382363 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:53.381922    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z79d" event={"ID":"8aa9591a-74d2-4890-be3f-482e458bd651","Type":"ContainerDied","Data":"78d1af60b5c8d314e0bf21138960ff3917db404674c7032692a2ceb17e0b5c4d"}
Jul 10 02:07:54.236723 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:54.236694    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:54.236887 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:54.236768    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:54.236887 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:54.236819    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:54.236976 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:54.236909    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:54.385450 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:54.385414    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw" event={"ID":"c541937e-9bfd-48fe-a99f-c97f0e4780f6","Type":"ContainerStarted","Data":"f01bf37addaffe77066afbdb05afa008dac62deacc7c8c674d84edbb91b58d21"}
Jul 10 02:07:54.388392 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:54.388371    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:07:54.388684 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:54.388664    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs" event={"ID":"cbeea170-d687-476c-b932-01800b00ecae","Type":"ContainerStarted","Data":"7e411224c1402ef34174908c564fc81cf0f7003c128f8956119034b76d39963d"}
Jul 10 02:07:54.408192 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:54.405953    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-dldnw" podStartSLOduration=1.328627805 podStartE2EDuration="29.405932556s" totalImagesPullingTime="28.077304751s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=3 observedRunningTime="2026-07-10 02:07:54.405355738 +0000 UTC m=+30.721665610" watchObservedRunningTime="2026-07-10 02:07:54.405932556 +0000 UTC m=+30.722242429"
Jul 10 02:07:55.236697 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:55.236655    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:55.236922 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:55.236778    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:55.392485 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:55.392456    2557 generic.go:356] "Generic (PLEG): container finished" podID="8aa9591a-74d2-4890-be3f-482e458bd651" containerID="7db9adf7bf78b9c5112cb479d19539a6014c378983449d218c6ed9182dda5b2f" exitCode=0
Jul 10 02:07:55.392869 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:55.392546    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z79d" event={"ID":"8aa9591a-74d2-4890-be3f-482e458bd651","Type":"ContainerDied","Data":"7db9adf7bf78b9c5112cb479d19539a6014c378983449d218c6ed9182dda5b2f"}
Jul 10 02:07:56.235981 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:56.235736    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:56.236188 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:56.235794    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:56.236188 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:56.236106    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:56.236188 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:56.236175    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:56.398396 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:56.398369    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:07:56.398964 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:56.398814    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs" event={"ID":"cbeea170-d687-476c-b932-01800b00ecae","Type":"ContainerStarted","Data":"8617925a5d8fcc313bcf55758c44db89d31c6b466245c007f2f92cfbea67cc3c"}
Jul 10 02:07:56.399095 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:56.399079    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:56.399291 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:56.399272    2557 scope.go:117] "RemoveContainer" containerID="b95a907d9defe1f02c36813209db92b173818dd6763974e45db9376f83a75d68"
Jul 10 02:07:56.417827 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:56.417787    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:56.960346 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:56.960306    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:56.960481 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:56.960354    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:56.960481 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:56.960443    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:56.960481 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:56.960443    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:56.960594 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:56.960495    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs podName:1a790021-08d0-4e49-8da8-f046861c72ac nodeName:}" failed. No retries permitted until 2026-07-10 02:08:28.960481986 +0000 UTC m=+65.276791839 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs") pod "network-metrics-daemon-5j8nl" (UID: "1a790021-08d0-4e49-8da8-f046861c72ac") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 10 02:07:56.960594 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:56.960512    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret podName:796726ff-5fe1-48dd-9ea0-e5fae11d44e2 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:28.960502758 +0000 UTC m=+65.276812615 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret") pod "global-pull-secret-syncer-mb7lt" (UID: "796726ff-5fe1-48dd-9ea0-e5fae11d44e2") : object "kube-system"/"original-pull-secret" not registered
Jul 10 02:07:57.061309 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.061278    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsqgn\" (UniqueName: \"kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn\") pod \"network-check-target-st5kh\" (UID: \"a85578ca-e7d0-4036-b2c0-8315bf972f66\") " pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:57.061458 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:57.061409    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 10 02:07:57.061458 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:57.061425    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 10 02:07:57.061458 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:57.061433    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fsqgn for pod openshift-network-diagnostics/network-check-target-st5kh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:57.061583 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:57.061489    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn podName:a85578ca-e7d0-4036-b2c0-8315bf972f66 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:29.061475405 +0000 UTC m=+65.377785275 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fsqgn" (UniqueName: "kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn") pod "network-check-target-st5kh" (UID: "a85578ca-e7d0-4036-b2c0-8315bf972f66") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 10 02:07:57.236719 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.236652    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:57.236880 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:57.236757    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:57.406137 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.406107    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:07:57.406549 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.406502    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs" event={"ID":"cbeea170-d687-476c-b932-01800b00ecae","Type":"ContainerStarted","Data":"57ab7e1548200df0ea4cc8b5ba692449396ee9fe2729f2546e7012851bdee261"}
Jul 10 02:07:57.406787 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.406763    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:57.406935 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.406851    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:57.425208 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.425171    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:07:57.434354 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.434312    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs" podStartSLOduration=7.531856177 podStartE2EDuration="32.434299108s" totalImagesPullingTime="24.902442931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:07:57.433249687 +0000 UTC m=+33.749559594" watchObservedRunningTime="2026-07-10 02:07:57.434299108 +0000 UTC m=+33.750609001"
Jul 10 02:07:57.436530 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.436504    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mb7lt"]
Jul 10 02:07:57.436632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.436599    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:57.436712 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:57.436693    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:07:57.439596 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.439574    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5j8nl"]
Jul 10 02:07:57.439731 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.439682    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:57.439847 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:57.439790    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:57.442515 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.442490    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-st5kh"]
Jul 10 02:07:57.442602 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:57.442585    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:57.442696 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:57.442670    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:59.236049 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:59.235997    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:07:59.236645 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:59.235997    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:07:59.236645 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:59.236112    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:07:59.236645 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:59.236198    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:07:59.236645 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:07:59.236002    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:07:59.236645 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:07:59.236269    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:08:01.236395 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:01.236355    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:08:01.236737 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:01.236461    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:08:01.236737 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:01.236485    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:08:01.236737 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:01.236479    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mb7lt" podUID="796726ff-5fe1-48dd-9ea0-e5fae11d44e2"
Jul 10 02:08:01.236737 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:01.236546    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-st5kh" podUID="a85578ca-e7d0-4036-b2c0-8315bf972f66"
Jul 10 02:08:01.236737 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:01.236636    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:08:02.423057 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:02.422889    2557 generic.go:356] "Generic (PLEG): container finished" podID="8aa9591a-74d2-4890-be3f-482e458bd651" containerID="c826d4643c9e99b6afdf7256dd1c1792891601de0df91d729354fa8f37a4577d" exitCode=0
Jul 10 02:08:02.423435 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:02.422973    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z79d" event={"ID":"8aa9591a-74d2-4890-be3f-482e458bd651","Type":"ContainerDied","Data":"c826d4643c9e99b6afdf7256dd1c1792891601de0df91d729354fa8f37a4577d"}
Jul 10 02:08:02.995236 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:02.995208    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-213.ec2.internal" event="NodeReady"
Jul 10 02:08:02.995400 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:02.995354    2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 10 02:08:03.034691 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.034655    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"]
Jul 10 02:08:03.046959 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.046936    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f5cf64794-c75bp"]
Jul 10 02:08:03.047095 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.047057    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:03.049289 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.049216    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-58c2q\""
Jul 10 02:08:03.049289 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.049228    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 10 02:08:03.049289 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.049218    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 10 02:08:03.062781 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.062760    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"]
Jul 10 02:08:03.062915 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.062793    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rv9wg"]
Jul 10 02:08:03.062915 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.062889    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.064640 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.064616    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 10 02:08:03.064759 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.064684    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g79n9\""
Jul 10 02:08:03.064759 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.064744    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 10 02:08:03.064904 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.064890    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 10 02:08:03.071247 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.071225    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 10 02:08:03.080756 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.080738    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f5cf64794-c75bp"]
Jul 10 02:08:03.080866 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.080766    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vsgwk"]
Jul 10 02:08:03.080919 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.080879    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rv9wg"
Jul 10 02:08:03.083936 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.083917    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 10 02:08:03.084041 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.083975    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 10 02:08:03.084210 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.084190    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 10 02:08:03.084302 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.084226    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zpbsc\""
Jul 10 02:08:03.106927 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.106907    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rv9wg"]
Jul 10 02:08:03.107038 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.106930    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.107038 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.106955    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vsgwk"]
Jul 10 02:08:03.108828 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.108775    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 10 02:08:03.108921 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.108857    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 10 02:08:03.109047 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.109035    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g8kf5\""
Jul 10 02:08:03.208088 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208003    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd2ph\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-kube-api-access-dd2ph\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.208088 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208040    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-bound-sa-token\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.208088 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208064    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d480e3b6-008b-433b-8a35-6d023ed58eff-tmp-dir\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.208088 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208089    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7008ee73-8f48-4845-89be-7c2a76aa76d7-ca-trust-extracted\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.208384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208130    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d480e3b6-008b-433b-8a35-6d023ed58eff-config-volume\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.208384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208165    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.208384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208187    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgsxl\" (UniqueName: \"kubernetes.io/projected/d480e3b6-008b-433b-8a35-6d023ed58eff-kube-api-access-rgsxl\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.208384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208209    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.208384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208226    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d86f5caf-4a63-4947-b175-ae4a035dd52b-cert\") pod \"ingress-canary-rv9wg\" (UID: \"d86f5caf-4a63-4947-b175-ae4a035dd52b\") " pod="openshift-ingress-canary/ingress-canary-rv9wg"
Jul 10 02:08:03.208384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208240    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/109d498f-fabd-4b87-b893-19dc87cb1f23-nginx-conf\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:03.208384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208256    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:03.208384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208335    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-image-registry-private-configuration\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.208384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208373    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmp8j\" (UniqueName: \"kubernetes.io/projected/d86f5caf-4a63-4947-b175-ae4a035dd52b-kube-api-access-rmp8j\") pod \"ingress-canary-rv9wg\" (UID: \"d86f5caf-4a63-4947-b175-ae4a035dd52b\") " pod="openshift-ingress-canary/ingress-canary-rv9wg"
Jul 10 02:08:03.208665 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208425    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-certificates\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.208665 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208442    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-installation-pull-secrets\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.208665 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.208468    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-trusted-ca\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.236197 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.236173    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:08:03.236303 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.236290    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:08:03.236340 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.236320    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:08:03.238756 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.238719    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 02:08:03.238909 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.238719    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 02:08:03.238909 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.238773    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dzm46\""
Jul 10 02:08:03.238909 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.238783    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-chbtx\""
Jul 10 02:08:03.238909 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.238798    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 02:08:03.239098 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.239080    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 02:08:03.308853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.308789    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd2ph\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-kube-api-access-dd2ph\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.308985 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.308948    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-bound-sa-token\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.308985 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.308979    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d480e3b6-008b-433b-8a35-6d023ed58eff-tmp-dir\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.309069 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309007    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7008ee73-8f48-4845-89be-7c2a76aa76d7-ca-trust-extracted\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.309069 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309024    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d480e3b6-008b-433b-8a35-6d023ed58eff-config-volume\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.309069 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309060    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.309207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309083    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgsxl\" (UniqueName: \"kubernetes.io/projected/d480e3b6-008b-433b-8a35-6d023ed58eff-kube-api-access-rgsxl\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.309207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309114    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.309207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309142    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d86f5caf-4a63-4947-b175-ae4a035dd52b-cert\") pod \"ingress-canary-rv9wg\" (UID: \"d86f5caf-4a63-4947-b175-ae4a035dd52b\") " pod="openshift-ingress-canary/ingress-canary-rv9wg"
Jul 10 02:08:03.309207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309172    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/109d498f-fabd-4b87-b893-19dc87cb1f23-nginx-conf\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:03.309207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309188    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:03.309439 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309215    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-image-registry-private-configuration\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.309439 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309236    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmp8j\" (UniqueName: \"kubernetes.io/projected/d86f5caf-4a63-4947-b175-ae4a035dd52b-kube-api-access-rmp8j\") pod \"ingress-canary-rv9wg\" (UID: \"d86f5caf-4a63-4947-b175-ae4a035dd52b\") " pod="openshift-ingress-canary/ingress-canary-rv9wg"
Jul 10 02:08:03.309439 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309275    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-certificates\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.309439 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309290    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-installation-pull-secrets\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.309439 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309312    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-trusted-ca\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.309439 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309336    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d480e3b6-008b-433b-8a35-6d023ed58eff-tmp-dir\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.309439 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.309421    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:08:03.309747 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.309485    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls podName:d480e3b6-008b-433b-8a35-6d023ed58eff nodeName:}" failed. No retries permitted until 2026-07-10 02:08:03.809464906 +0000 UTC m=+40.125774757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls") pod "dns-default-vsgwk" (UID: "d480e3b6-008b-433b-8a35-6d023ed58eff") : secret "dns-default-metrics-tls" not found
Jul 10 02:08:03.309747 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.309480    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7008ee73-8f48-4845-89be-7c2a76aa76d7-ca-trust-extracted\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.309747 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.309502    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:08:03.309747 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.309519    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5cf64794-c75bp: secret "image-registry-tls" not found
Jul 10 02:08:03.310353 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.310328    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d480e3b6-008b-433b-8a35-6d023ed58eff-config-volume\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.310445 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.310351    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/109d498f-fabd-4b87-b893-19dc87cb1f23-nginx-conf\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:03.310445 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.310293    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-certificates\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.310639 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.310624    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls podName:7008ee73-8f48-4845-89be-7c2a76aa76d7 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:03.809550395 +0000 UTC m=+40.125860264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls") pod "image-registry-5f5cf64794-c75bp" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7") : secret "image-registry-tls" not found
Jul 10 02:08:03.312226 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.310841    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:08:03.312226 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.310945    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-trusted-ca\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.312226 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.311743    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert podName:109d498f-fabd-4b87-b893-19dc87cb1f23 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:03.811714768 +0000 UTC m=+40.128024620 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rr7f6" (UID: "109d498f-fabd-4b87-b893-19dc87cb1f23") : secret "networking-console-plugin-cert" not found
Jul 10 02:08:03.315222 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.315198    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d86f5caf-4a63-4947-b175-ae4a035dd52b-cert\") pod \"ingress-canary-rv9wg\" (UID: \"d86f5caf-4a63-4947-b175-ae4a035dd52b\") " pod="openshift-ingress-canary/ingress-canary-rv9wg"
Jul 10 02:08:03.315358 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.315265    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-image-registry-private-configuration\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.315517 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.315495    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-installation-pull-secrets\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.318994 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.318963    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-bound-sa-token\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.319094 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.319001    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgsxl\" (UniqueName: \"kubernetes.io/projected/d480e3b6-008b-433b-8a35-6d023ed58eff-kube-api-access-rgsxl\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.319160 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.319124    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmp8j\" (UniqueName: \"kubernetes.io/projected/d86f5caf-4a63-4947-b175-ae4a035dd52b-kube-api-access-rmp8j\") pod \"ingress-canary-rv9wg\" (UID: \"d86f5caf-4a63-4947-b175-ae4a035dd52b\") " pod="openshift-ingress-canary/ingress-canary-rv9wg"
Jul 10 02:08:03.319500 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.319484    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd2ph\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-kube-api-access-dd2ph\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.389489 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.389466    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rv9wg"
Jul 10 02:08:03.428422 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.428393    2557 generic.go:356] "Generic (PLEG): container finished" podID="8aa9591a-74d2-4890-be3f-482e458bd651" containerID="df032472b81483904a4664b7ca9b82f08dd7b3951e83aad9e015e42f431a48b1" exitCode=0
Jul 10 02:08:03.428969 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.428450    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z79d" event={"ID":"8aa9591a-74d2-4890-be3f-482e458bd651","Type":"ContainerDied","Data":"df032472b81483904a4664b7ca9b82f08dd7b3951e83aad9e015e42f431a48b1"}
Jul 10 02:08:03.593596 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.593387    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rv9wg"]
Jul 10 02:08:03.597119 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:08:03.597096    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd86f5caf_4a63_4947_b175_ae4a035dd52b.slice/crio-feb3473d8ab23d2a8c0ff47cea348068620cd8ab3f72e5b4aeca21e5ab0ce6c1 WatchSource:0}: Error finding container feb3473d8ab23d2a8c0ff47cea348068620cd8ab3f72e5b4aeca21e5ab0ce6c1: Status 404 returned error can't find the container with id feb3473d8ab23d2a8c0ff47cea348068620cd8ab3f72e5b4aeca21e5ab0ce6c1
Jul 10 02:08:03.813430 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.813400    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:03.813594 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.813491    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:03.813594 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:03.813510    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:03.813594 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.813560    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:08:03.813594 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.813590    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:08:03.813706 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.813600    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5cf64794-c75bp: secret "image-registry-tls" not found
Jul 10 02:08:03.813706 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.813620    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:08:03.813706 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.813642    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert podName:109d498f-fabd-4b87-b893-19dc87cb1f23 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:04.813620282 +0000 UTC m=+41.129930136 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rr7f6" (UID: "109d498f-fabd-4b87-b893-19dc87cb1f23") : secret "networking-console-plugin-cert" not found
Jul 10 02:08:03.813706 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.813660    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls podName:7008ee73-8f48-4845-89be-7c2a76aa76d7 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:04.813652831 +0000 UTC m=+41.129962682 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls") pod "image-registry-5f5cf64794-c75bp" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7") : secret "image-registry-tls" not found
Jul 10 02:08:03.813706 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:03.813669    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls podName:d480e3b6-008b-433b-8a35-6d023ed58eff nodeName:}" failed. No retries permitted until 2026-07-10 02:08:04.813664641 +0000 UTC m=+41.129974490 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls") pod "dns-default-vsgwk" (UID: "d480e3b6-008b-433b-8a35-6d023ed58eff") : secret "dns-default-metrics-tls" not found
Jul 10 02:08:04.431889 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:04.431855    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rv9wg" event={"ID":"d86f5caf-4a63-4947-b175-ae4a035dd52b","Type":"ContainerStarted","Data":"feb3473d8ab23d2a8c0ff47cea348068620cd8ab3f72e5b4aeca21e5ab0ce6c1"}
Jul 10 02:08:04.435130 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:04.435101    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7z79d" event={"ID":"8aa9591a-74d2-4890-be3f-482e458bd651","Type":"ContainerStarted","Data":"de70f0ad73eb6c34dfdda88c69985ef1ccac35fba30db68a86141dbcedd7b161"}
Jul 10 02:08:04.461483 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:04.461431    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7z79d" podStartSLOduration=8.100509791 podStartE2EDuration="39.461411304s" totalImagesPullingTime="31.360901513s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:08:04.460827525 +0000 UTC m=+40.777137402" watchObservedRunningTime="2026-07-10 02:08:04.461411304 +0000 UTC m=+40.777721154"
Jul 10 02:08:04.821187 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:04.821149    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:04.821378 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:04.821199    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:04.821378 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:04.821321    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:08:04.821503 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:04.821378    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:08:04.821503 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:04.821391    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls podName:d480e3b6-008b-433b-8a35-6d023ed58eff nodeName:}" failed. No retries permitted until 2026-07-10 02:08:06.821370667 +0000 UTC m=+43.137680519 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls") pod "dns-default-vsgwk" (UID: "d480e3b6-008b-433b-8a35-6d023ed58eff") : secret "dns-default-metrics-tls" not found
Jul 10 02:08:04.821503 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:04.821394    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5cf64794-c75bp: secret "image-registry-tls" not found
Jul 10 02:08:04.821503 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:04.821417    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:04.821503 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:04.821445    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls podName:7008ee73-8f48-4845-89be-7c2a76aa76d7 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:06.821427988 +0000 UTC m=+43.137737841 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls") pod "image-registry-5f5cf64794-c75bp" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7") : secret "image-registry-tls" not found
Jul 10 02:08:04.821503 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:04.821481    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:08:04.821713 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:04.821518    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert podName:109d498f-fabd-4b87-b893-19dc87cb1f23 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:06.821507022 +0000 UTC m=+43.137816886 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rr7f6" (UID: "109d498f-fabd-4b87-b893-19dc87cb1f23") : secret "networking-console-plugin-cert" not found
Jul 10 02:08:06.840552 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:06.840510    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:06.840552 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:06.840562    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:06.841144 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:06.840596    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:06.841144 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:06.840670    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:08:06.841144 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:06.840734    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls podName:d480e3b6-008b-433b-8a35-6d023ed58eff nodeName:}" failed. No retries permitted until 2026-07-10 02:08:10.840715801 +0000 UTC m=+47.157025670 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls") pod "dns-default-vsgwk" (UID: "d480e3b6-008b-433b-8a35-6d023ed58eff") : secret "dns-default-metrics-tls" not found
Jul 10 02:08:06.841144 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:06.840763    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:08:06.841144 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:06.840788    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5cf64794-c75bp: secret "image-registry-tls" not found
Jul 10 02:08:06.841144 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:06.840796    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:08:06.841144 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:06.840851    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert podName:109d498f-fabd-4b87-b893-19dc87cb1f23 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:10.840839159 +0000 UTC m=+47.157149023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rr7f6" (UID: "109d498f-fabd-4b87-b893-19dc87cb1f23") : secret "networking-console-plugin-cert" not found
Jul 10 02:08:06.841144 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:06.840867    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls podName:7008ee73-8f48-4845-89be-7c2a76aa76d7 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:10.840858763 +0000 UTC m=+47.157168613 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls") pod "image-registry-5f5cf64794-c75bp" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7") : secret "image-registry-tls" not found
Jul 10 02:08:08.445373 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:08.445333    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rv9wg" event={"ID":"d86f5caf-4a63-4947-b175-ae4a035dd52b","Type":"ContainerStarted","Data":"58d026d61fe89d1e3f6cbb079dce283ea6a9b5dd8cd558633875d81e02378f3f"}
Jul 10 02:08:08.462185 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:08.462136    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rv9wg" podStartSLOduration=1.582803131 podStartE2EDuration="5.462120972s" totalImagesPullingTime="3.879317841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:08:08.461680429 +0000 UTC m=+44.777990301" watchObservedRunningTime="2026-07-10 02:08:08.462120972 +0000 UTC m=+44.778430843"
Jul 10 02:08:09.418543 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.418376    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"]
Jul 10 02:08:09.442304 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.442277    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"]
Jul 10 02:08:09.442457 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.442386    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:09.444289 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.444269    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 10 02:08:09.444414 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.444323    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 10 02:08:09.444414 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.444374    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 10 02:08:09.444539 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.444419    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 10 02:08:09.454320 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.454290    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"]
Jul 10 02:08:09.454579 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.454381    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"
Jul 10 02:08:09.456075 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.456056    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-8m9s2\""
Jul 10 02:08:09.456168 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.456059    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 10 02:08:09.472923 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.472893    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"]
Jul 10 02:08:09.473082 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.472942    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"]
Jul 10 02:08:09.473082 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.472957    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"]
Jul 10 02:08:09.473082 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.473060    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.477198 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.476749    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 10 02:08:09.477198 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.477021    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 10 02:08:09.477198 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.477056    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 10 02:08:09.477428 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.477290    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 10 02:08:09.561087 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.561046    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c12cbd8d-4e9c-44cb-a769-f34924a81c46-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx\" (UID: \"c12cbd8d-4e9c-44cb-a769-f34924a81c46\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"
Jul 10 02:08:09.561250 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.561103    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0444260c-a7df-4444-960a-c077b5a09f8f-klusterlet-config\") pod \"klusterlet-addon-workmgr-f944b9f5-mmttl\" (UID: \"0444260c-a7df-4444-960a-c077b5a09f8f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:09.561250 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.561203    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0444260c-a7df-4444-960a-c077b5a09f8f-tmp\") pod \"klusterlet-addon-workmgr-f944b9f5-mmttl\" (UID: \"0444260c-a7df-4444-960a-c077b5a09f8f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:09.561250 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.561231    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4msc9\" (UniqueName: \"kubernetes.io/projected/c12cbd8d-4e9c-44cb-a769-f34924a81c46-kube-api-access-4msc9\") pod \"managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx\" (UID: \"c12cbd8d-4e9c-44cb-a769-f34924a81c46\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"
Jul 10 02:08:09.561405 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.561297    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmcgq\" (UniqueName: \"kubernetes.io/projected/0444260c-a7df-4444-960a-c077b5a09f8f-kube-api-access-gmcgq\") pod \"klusterlet-addon-workmgr-f944b9f5-mmttl\" (UID: \"0444260c-a7df-4444-960a-c077b5a09f8f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:09.661692 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.661655    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0444260c-a7df-4444-960a-c077b5a09f8f-tmp\") pod \"klusterlet-addon-workmgr-f944b9f5-mmttl\" (UID: \"0444260c-a7df-4444-960a-c077b5a09f8f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:09.661692 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.661695    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmcgq\" (UniqueName: \"kubernetes.io/projected/0444260c-a7df-4444-960a-c077b5a09f8f-kube-api-access-gmcgq\") pod \"klusterlet-addon-workmgr-f944b9f5-mmttl\" (UID: \"0444260c-a7df-4444-960a-c077b5a09f8f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:09.661916 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.661719    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.661916 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.661748    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c12cbd8d-4e9c-44cb-a769-f34924a81c46-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx\" (UID: \"c12cbd8d-4e9c-44cb-a769-f34924a81c46\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"
Jul 10 02:08:09.661916 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.661770    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.661916 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.661851    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0444260c-a7df-4444-960a-c077b5a09f8f-klusterlet-config\") pod \"klusterlet-addon-workmgr-f944b9f5-mmttl\" (UID: \"0444260c-a7df-4444-960a-c077b5a09f8f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:09.661916 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.661906    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-ca\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.662170 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.661960    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnz25\" (UniqueName: \"kubernetes.io/projected/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-kube-api-access-cnz25\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.662170 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.662019    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.662170 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.662072    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4msc9\" (UniqueName: \"kubernetes.io/projected/c12cbd8d-4e9c-44cb-a769-f34924a81c46-kube-api-access-4msc9\") pod \"managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx\" (UID: \"c12cbd8d-4e9c-44cb-a769-f34924a81c46\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"
Jul 10 02:08:09.662170 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.662096    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-hub\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.662170 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.662100    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0444260c-a7df-4444-960a-c077b5a09f8f-tmp\") pod \"klusterlet-addon-workmgr-f944b9f5-mmttl\" (UID: \"0444260c-a7df-4444-960a-c077b5a09f8f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:09.665788 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.665763    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/0444260c-a7df-4444-960a-c077b5a09f8f-klusterlet-config\") pod \"klusterlet-addon-workmgr-f944b9f5-mmttl\" (UID: \"0444260c-a7df-4444-960a-c077b5a09f8f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:09.665788 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.665773    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c12cbd8d-4e9c-44cb-a769-f34924a81c46-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx\" (UID: \"c12cbd8d-4e9c-44cb-a769-f34924a81c46\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"
Jul 10 02:08:09.680556 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.680488    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4msc9\" (UniqueName: \"kubernetes.io/projected/c12cbd8d-4e9c-44cb-a769-f34924a81c46-kube-api-access-4msc9\") pod \"managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx\" (UID: \"c12cbd8d-4e9c-44cb-a769-f34924a81c46\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"
Jul 10 02:08:09.680681 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.680566    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmcgq\" (UniqueName: \"kubernetes.io/projected/0444260c-a7df-4444-960a-c077b5a09f8f-kube-api-access-gmcgq\") pod \"klusterlet-addon-workmgr-f944b9f5-mmttl\" (UID: \"0444260c-a7df-4444-960a-c077b5a09f8f\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:09.753522 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.753488    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:09.762792 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.762764    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.762900 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.762848    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-hub\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.762900 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.762891    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.762995 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.762921    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.762995 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.762980    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-ca\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.763170 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.763140    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnz25\" (UniqueName: \"kubernetes.io/projected/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-kube-api-access-cnz25\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.763716 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.763684    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-additional-service-ca\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.765773 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.765750    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-ca\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.765875 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.765755    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.765875 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.765823    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-hub\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.765875 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.765825    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.772301 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.772277    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnz25\" (UniqueName: \"kubernetes.io/projected/df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb-kube-api-access-cnz25\") pod \"cluster-proxy-proxy-agent-86ccb77c9d-dx6mt\" (UID: \"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.772407 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.772394    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"
Jul 10 02:08:09.788271 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.788243    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:08:09.908447 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.908417    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"]
Jul 10 02:08:09.912401 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:08:09.912374    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0444260c_a7df_4444_960a_c077b5a09f8f.slice/crio-8cad9f958383a6bcbed8f9730830a4f44f559316f713fa8e327eb8417a68a0d9 WatchSource:0}: Error finding container 8cad9f958383a6bcbed8f9730830a4f44f559316f713fa8e327eb8417a68a0d9: Status 404 returned error can't find the container with id 8cad9f958383a6bcbed8f9730830a4f44f559316f713fa8e327eb8417a68a0d9
Jul 10 02:08:09.944644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.944568    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"]
Jul 10 02:08:09.948359 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:08:09.948326    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc12cbd8d_4e9c_44cb_a769_f34924a81c46.slice/crio-04522090f8de05c58b2fc56c08b722deb325b365aa094a4e5a0fac5e85361543 WatchSource:0}: Error finding container 04522090f8de05c58b2fc56c08b722deb325b365aa094a4e5a0fac5e85361543: Status 404 returned error can't find the container with id 04522090f8de05c58b2fc56c08b722deb325b365aa094a4e5a0fac5e85361543
Jul 10 02:08:09.969768 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:09.969743    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"]
Jul 10 02:08:09.979131 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:08:09.979106    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf4d97b8_bff0_4ab2_8aa7_3ad14dd5efbb.slice/crio-e9d247fe7339e26b48485c874c98f6763d6e5b41c3475625677cb1d271f7efa8 WatchSource:0}: Error finding container e9d247fe7339e26b48485c874c98f6763d6e5b41c3475625677cb1d271f7efa8: Status 404 returned error can't find the container with id e9d247fe7339e26b48485c874c98f6763d6e5b41c3475625677cb1d271f7efa8
Jul 10 02:08:10.451043 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:10.450998    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx" event={"ID":"c12cbd8d-4e9c-44cb-a769-f34924a81c46","Type":"ContainerStarted","Data":"04522090f8de05c58b2fc56c08b722deb325b365aa094a4e5a0fac5e85361543"}
Jul 10 02:08:10.452229 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:10.452195    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl" event={"ID":"0444260c-a7df-4444-960a-c077b5a09f8f","Type":"ContainerStarted","Data":"8cad9f958383a6bcbed8f9730830a4f44f559316f713fa8e327eb8417a68a0d9"}
Jul 10 02:08:10.453267 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:10.453237    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" event={"ID":"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb","Type":"ContainerStarted","Data":"e9d247fe7339e26b48485c874c98f6763d6e5b41c3475625677cb1d271f7efa8"}
Jul 10 02:08:10.872207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:10.872161    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:10.873233 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:10.872217    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:10.873233 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:10.872275    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:10.873233 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:10.872429    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:08:10.873233 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:10.872489    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls podName:d480e3b6-008b-433b-8a35-6d023ed58eff nodeName:}" failed. No retries permitted until 2026-07-10 02:08:18.87247038 +0000 UTC m=+55.188780244 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls") pod "dns-default-vsgwk" (UID: "d480e3b6-008b-433b-8a35-6d023ed58eff") : secret "dns-default-metrics-tls" not found
Jul 10 02:08:10.873233 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:10.872653    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:08:10.873233 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:10.872723    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert podName:109d498f-fabd-4b87-b893-19dc87cb1f23 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:18.872704371 +0000 UTC m=+55.189014227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rr7f6" (UID: "109d498f-fabd-4b87-b893-19dc87cb1f23") : secret "networking-console-plugin-cert" not found
Jul 10 02:08:10.873233 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:10.872757    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:08:10.873233 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:10.872776    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5cf64794-c75bp: secret "image-registry-tls" not found
Jul 10 02:08:10.873233 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:10.872864    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls podName:7008ee73-8f48-4845-89be-7c2a76aa76d7 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:18.872847741 +0000 UTC m=+55.189157606 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls") pod "image-registry-5f5cf64794-c75bp" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7") : secret "image-registry-tls" not found
Jul 10 02:08:17.475326 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:17.475290    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl" event={"ID":"0444260c-a7df-4444-960a-c077b5a09f8f","Type":"ContainerStarted","Data":"19b48ec41be07947c086dbbe81a1a7ce855c98254dc6122a5eb67e8b74d0f8c2"}
Jul 10 02:08:17.475766 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:17.475489    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:17.477179 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:17.477152    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:08:17.477696 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:17.477635    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" event={"ID":"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb","Type":"ContainerStarted","Data":"c765594265f9344a5297653c7231ab4240a8dcbfdb3581ebad5bab41a5927be8"}
Jul 10 02:08:17.477799 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:17.477706    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" event={"ID":"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb","Type":"ContainerStarted","Data":"706db1d545e5d4da4cc7c79a896ab1ac855a0f960043d0d7de00ba8cdc8577ad"}
Jul 10 02:08:17.477799 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:17.477720    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" event={"ID":"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb","Type":"ContainerStarted","Data":"10b5fc970b80899fbc9390f51e9f468766116eeb2b02030dad90597cbba74f22"}
Jul 10 02:08:17.479155 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:17.479136    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx" event={"ID":"c12cbd8d-4e9c-44cb-a769-f34924a81c46","Type":"ContainerStarted","Data":"ebae43471c61a90c99f3986b8b71b27ec8738cc3f86684b4c54c494ba98ded30"}
Jul 10 02:08:17.508640 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:17.508588    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl" podStartSLOduration=2.008434352 podStartE2EDuration="8.508573603s" totalImagesPullingTime="6.500139251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:08:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:08:17.491704048 +0000 UTC m=+53.808013959" watchObservedRunningTime="2026-07-10 02:08:17.508573603 +0000 UTC m=+53.824883474"
Jul 10 02:08:17.527446 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:17.527391    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx" podStartSLOduration=2.062024847 podStartE2EDuration="8.527373556s" totalImagesPullingTime="6.465348709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:08:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:08:17.527209807 +0000 UTC m=+53.843519681" watchObservedRunningTime="2026-07-10 02:08:17.527373556 +0000 UTC m=+53.843683431"
Jul 10 02:08:17.545816 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:17.545764    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" podStartSLOduration=2.096295459 podStartE2EDuration="8.545750443s" totalImagesPullingTime="6.449454984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:08:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:08:17.545094388 +0000 UTC m=+53.861404261" watchObservedRunningTime="2026-07-10 02:08:17.545750443 +0000 UTC m=+53.862060314"
Jul 10 02:08:18.941577 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:18.941537    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:18.941577 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:18.941579    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:18.942083 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:18.941655    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:18.942083 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:18.941685    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:08:18.942083 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:18.941700    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5cf64794-c75bp: secret "image-registry-tls" not found
Jul 10 02:08:18.942083 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:18.941732    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:08:18.942083 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:18.941748    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:08:18.942083 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:18.941764    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls podName:7008ee73-8f48-4845-89be-7c2a76aa76d7 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:34.941744647 +0000 UTC m=+71.258054520 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls") pod "image-registry-5f5cf64794-c75bp" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7") : secret "image-registry-tls" not found
Jul 10 02:08:18.942083 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:18.941783    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert podName:109d498f-fabd-4b87-b893-19dc87cb1f23 nodeName:}" failed. No retries permitted until 2026-07-10 02:08:34.941773441 +0000 UTC m=+71.258083310 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rr7f6" (UID: "109d498f-fabd-4b87-b893-19dc87cb1f23") : secret "networking-console-plugin-cert" not found
Jul 10 02:08:18.942083 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:18.941830    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls podName:d480e3b6-008b-433b-8a35-6d023ed58eff nodeName:}" failed. No retries permitted until 2026-07-10 02:08:34.941798085 +0000 UTC m=+71.258107934 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls") pod "dns-default-vsgwk" (UID: "d480e3b6-008b-433b-8a35-6d023ed58eff") : secret "dns-default-metrics-tls" not found
Jul 10 02:08:29.016941 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.016890    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:08:29.016941 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.016948    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:08:29.019055 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.019034    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 10 02:08:29.019120 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.019064    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 10 02:08:29.027671 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:29.027652    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 02:08:29.027717 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:29.027708    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs podName:1a790021-08d0-4e49-8da8-f046861c72ac nodeName:}" failed. No retries permitted until 2026-07-10 02:09:33.027693768 +0000 UTC m=+129.344003621 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs") pod "network-metrics-daemon-5j8nl" (UID: "1a790021-08d0-4e49-8da8-f046861c72ac") : secret "metrics-daemon-secret" not found
Jul 10 02:08:29.029883 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.029853    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/796726ff-5fe1-48dd-9ea0-e5fae11d44e2-original-pull-secret\") pod \"global-pull-secret-syncer-mb7lt\" (UID: \"796726ff-5fe1-48dd-9ea0-e5fae11d44e2\") " pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:08:29.055244 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.055219    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mb7lt"
Jul 10 02:08:29.118134 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.118049    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsqgn\" (UniqueName: \"kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn\") pod \"network-check-target-st5kh\" (UID: \"a85578ca-e7d0-4036-b2c0-8315bf972f66\") " pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:08:29.120737 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.120370    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 10 02:08:29.130566 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.130547    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 10 02:08:29.141287 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.141254    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsqgn\" (UniqueName: \"kubernetes.io/projected/a85578ca-e7d0-4036-b2c0-8315bf972f66-kube-api-access-fsqgn\") pod \"network-check-target-st5kh\" (UID: \"a85578ca-e7d0-4036-b2c0-8315bf972f66\") " pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:08:29.179302 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.179273    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mb7lt"]
Jul 10 02:08:29.353144 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.353119    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-chbtx\""
Jul 10 02:08:29.361506 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.361484    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:08:29.422553 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.422528    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-r4vzs"
Jul 10 02:08:29.484230 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.484202    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-st5kh"]
Jul 10 02:08:29.486941 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:08:29.486918    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda85578ca_e7d0_4036_b2c0_8315bf972f66.slice/crio-1578b7651c832657d628073986d079922d201025ec3dc6c8b42344c5941fee44 WatchSource:0}: Error finding container 1578b7651c832657d628073986d079922d201025ec3dc6c8b42344c5941fee44: Status 404 returned error can't find the container with id 1578b7651c832657d628073986d079922d201025ec3dc6c8b42344c5941fee44
Jul 10 02:08:29.512698 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.512669    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-st5kh" event={"ID":"a85578ca-e7d0-4036-b2c0-8315bf972f66","Type":"ContainerStarted","Data":"1578b7651c832657d628073986d079922d201025ec3dc6c8b42344c5941fee44"}
Jul 10 02:08:29.513664 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:29.513648    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mb7lt" event={"ID":"796726ff-5fe1-48dd-9ea0-e5fae11d44e2","Type":"ContainerStarted","Data":"bf8f8e9516d61121dfd2aa1a120a46cfbca4ba66550d79fd6647602d33a3d231"}
Jul 10 02:08:34.532050 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:34.532008    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-st5kh" event={"ID":"a85578ca-e7d0-4036-b2c0-8315bf972f66","Type":"ContainerStarted","Data":"e3456e925eea61bbb1ab6a99cd08e81ad8565c9a92149aa926c5fcc6a0725cca"}
Jul 10 02:08:34.532557 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:34.532132    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:08:34.533488 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:34.533464    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mb7lt" event={"ID":"796726ff-5fe1-48dd-9ea0-e5fae11d44e2","Type":"ContainerStarted","Data":"bc5147af5fb066433ae4e8fc9b59259b92a25a0cc604bc2ceeb0cc6aa2d914d3"}
Jul 10 02:08:34.548143 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:34.548093    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-st5kh" podStartSLOduration=64.662694983 podStartE2EDuration="1m9.548077951s" totalImagesPullingTime="4.885382968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:08:34.547890107 +0000 UTC m=+70.864200000" watchObservedRunningTime="2026-07-10 02:08:34.548077951 +0000 UTC m=+70.864387822"
Jul 10 02:08:34.564344 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:34.564278    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mb7lt" podStartSLOduration=64.36896067 podStartE2EDuration="1m9.564258166s" totalImagesPullingTime="5.195297496s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:08:34.562897035 +0000 UTC m=+70.879206928" watchObservedRunningTime="2026-07-10 02:08:34.564258166 +0000 UTC m=+70.880568038"
Jul 10 02:08:34.966861 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:34.966794    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:08:34.966861 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:34.966864    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:08:34.967080 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:08:34.966897    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:08:34.967080 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:34.966949    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:08:34.967080 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:34.966997    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:08:34.967080 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:34.967017    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert podName:109d498f-fabd-4b87-b893-19dc87cb1f23 nodeName:}" failed. No retries permitted until 2026-07-10 02:09:06.96700134 +0000 UTC m=+103.283311208 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rr7f6" (UID: "109d498f-fabd-4b87-b893-19dc87cb1f23") : secret "networking-console-plugin-cert" not found
Jul 10 02:08:34.967080 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:34.966955    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:08:34.967080 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:34.967050    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5cf64794-c75bp: secret "image-registry-tls" not found
Jul 10 02:08:34.967080 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:34.967034    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls podName:d480e3b6-008b-433b-8a35-6d023ed58eff nodeName:}" failed. No retries permitted until 2026-07-10 02:09:06.967024066 +0000 UTC m=+103.283333929 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls") pod "dns-default-vsgwk" (UID: "d480e3b6-008b-433b-8a35-6d023ed58eff") : secret "dns-default-metrics-tls" not found
Jul 10 02:08:34.967316 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:08:34.967085    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls podName:7008ee73-8f48-4845-89be-7c2a76aa76d7 nodeName:}" failed. No retries permitted until 2026-07-10 02:09:06.967076051 +0000 UTC m=+103.283385901 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls") pod "image-registry-5f5cf64794-c75bp" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7") : secret "image-registry-tls" not found
Jul 10 02:09:05.539336 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:09:05.539240    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-st5kh"
Jul 10 02:09:07.016056 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:09:07.016016    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:09:07.016056 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:09:07.016054    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:09:07.016476 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:09:07.016084    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:09:07.016476 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:09:07.016161    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:09:07.016476 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:09:07.016181    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5cf64794-c75bp: secret "image-registry-tls" not found
Jul 10 02:09:07.016476 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:09:07.016163    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:09:07.016476 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:09:07.016164    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:09:07.016476 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:09:07.016236    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls podName:7008ee73-8f48-4845-89be-7c2a76aa76d7 nodeName:}" failed. No retries permitted until 2026-07-10 02:10:11.016220734 +0000 UTC m=+167.332530585 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls") pod "image-registry-5f5cf64794-c75bp" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7") : secret "image-registry-tls" not found
Jul 10 02:09:07.016476 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:09:07.016257    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert podName:109d498f-fabd-4b87-b893-19dc87cb1f23 nodeName:}" failed. No retries permitted until 2026-07-10 02:10:11.016240896 +0000 UTC m=+167.332550746 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rr7f6" (UID: "109d498f-fabd-4b87-b893-19dc87cb1f23") : secret "networking-console-plugin-cert" not found
Jul 10 02:09:07.016476 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:09:07.016269    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls podName:d480e3b6-008b-433b-8a35-6d023ed58eff nodeName:}" failed. No retries permitted until 2026-07-10 02:10:11.016263572 +0000 UTC m=+167.332573422 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls") pod "dns-default-vsgwk" (UID: "d480e3b6-008b-433b-8a35-6d023ed58eff") : secret "dns-default-metrics-tls" not found
Jul 10 02:09:33.100427 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:09:33.100389    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:09:33.100992 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:09:33.100551    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 02:09:33.100992 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:09:33.100649    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs podName:1a790021-08d0-4e49-8da8-f046861c72ac nodeName:}" failed. No retries permitted until 2026-07-10 02:11:35.100626517 +0000 UTC m=+251.416936374 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs") pod "network-metrics-daemon-5j8nl" (UID: "1a790021-08d0-4e49-8da8-f046861c72ac") : secret "metrics-daemon-secret" not found
Jul 10 02:10:06.057294 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:10:06.057232    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6" podUID="109d498f-fabd-4b87-b893-19dc87cb1f23"
Jul 10 02:10:06.073044 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:10:06.073004    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp" podUID="7008ee73-8f48-4845-89be-7c2a76aa76d7"
Jul 10 02:10:06.115322 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:10:06.115285    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vsgwk" podUID="d480e3b6-008b-433b-8a35-6d023ed58eff"
Jul 10 02:10:06.245703 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:10:06.245668    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:10:06.802121 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:06.802084    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:10:06.802328 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:06.802084    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:10:06.802328 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:06.802088    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:10:11.085993 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:11.085950    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:10:11.085993 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:11.085995    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:10:11.086403 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:11.086030    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:10:11.086403 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:10:11.086110    2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 10 02:10:11.086403 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:10:11.086127    2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 10 02:10:11.086403 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:10:11.086131    2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f5cf64794-c75bp: secret "image-registry-tls" not found
Jul 10 02:10:11.086403 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:10:11.086127    2557 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 10 02:10:11.086403 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:10:11.086182    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls podName:d480e3b6-008b-433b-8a35-6d023ed58eff nodeName:}" failed. No retries permitted until 2026-07-10 02:12:13.086169178 +0000 UTC m=+289.402479042 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls") pod "dns-default-vsgwk" (UID: "d480e3b6-008b-433b-8a35-6d023ed58eff") : secret "dns-default-metrics-tls" not found
Jul 10 02:10:11.086403 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:10:11.086220    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls podName:7008ee73-8f48-4845-89be-7c2a76aa76d7 nodeName:}" failed. No retries permitted until 2026-07-10 02:12:13.086206192 +0000 UTC m=+289.402516045 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls") pod "image-registry-5f5cf64794-c75bp" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7") : secret "image-registry-tls" not found
Jul 10 02:10:11.086403 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:10:11.086236    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert podName:109d498f-fabd-4b87-b893-19dc87cb1f23 nodeName:}" failed. No retries permitted until 2026-07-10 02:12:13.086230269 +0000 UTC m=+289.402540119 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rr7f6" (UID: "109d498f-fabd-4b87-b893-19dc87cb1f23") : secret "networking-console-plugin-cert" not found
Jul 10 02:10:16.836192 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:16.836111    2557 generic.go:356] "Generic (PLEG): container finished" podID="c12cbd8d-4e9c-44cb-a769-f34924a81c46" containerID="ebae43471c61a90c99f3986b8b71b27ec8738cc3f86684b4c54c494ba98ded30" exitCode=255
Jul 10 02:10:16.836633 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:16.836185    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx" event={"ID":"c12cbd8d-4e9c-44cb-a769-f34924a81c46","Type":"ContainerDied","Data":"ebae43471c61a90c99f3986b8b71b27ec8738cc3f86684b4c54c494ba98ded30"}
Jul 10 02:10:16.836633 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:16.836526    2557 scope.go:117] "RemoveContainer" containerID="ebae43471c61a90c99f3986b8b71b27ec8738cc3f86684b4c54c494ba98ded30"
Jul 10 02:10:16.837651 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:16.837623    2557 generic.go:356] "Generic (PLEG): container finished" podID="0444260c-a7df-4444-960a-c077b5a09f8f" containerID="19b48ec41be07947c086dbbe81a1a7ce855c98254dc6122a5eb67e8b74d0f8c2" exitCode=1
Jul 10 02:10:16.837747 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:16.837660    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl" event={"ID":"0444260c-a7df-4444-960a-c077b5a09f8f","Type":"ContainerDied","Data":"19b48ec41be07947c086dbbe81a1a7ce855c98254dc6122a5eb67e8b74d0f8c2"}
Jul 10 02:10:16.837955 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:16.837943    2557 scope.go:117] "RemoveContainer" containerID="19b48ec41be07947c086dbbe81a1a7ce855c98254dc6122a5eb67e8b74d0f8c2"
Jul 10 02:10:17.475685 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:17.475646    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:10:17.842308 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:17.842276    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx" event={"ID":"c12cbd8d-4e9c-44cb-a769-f34924a81c46","Type":"ContainerStarted","Data":"277c63b7480508564b4dc5e1b7b80ef02bf901e3f90f0109414f97938f29eef4"}
Jul 10 02:10:17.844017 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:17.843989    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl" event={"ID":"0444260c-a7df-4444-960a-c077b5a09f8f","Type":"ContainerStarted","Data":"2710a1ea515af5ca39aa8ce45bc4505917e9db4fba11d6f6cb1241631763ebf8"}
Jul 10 02:10:17.844168 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:17.844149    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:10:17.844741 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:17.844728    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:10:21.236663 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:10:21.236630    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:11:35.196258 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:35.196206    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:11:35.196774 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:35.196375    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 10 02:11:35.196774 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:35.196460    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs podName:1a790021-08d0-4e49-8da8-f046861c72ac nodeName:}" failed. No retries permitted until 2026-07-10 02:13:37.196436653 +0000 UTC m=+373.512746520 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs") pod "network-metrics-daemon-5j8nl" (UID: "1a790021-08d0-4e49-8da8-f046861c72ac") : secret "metrics-daemon-secret" not found
Jul 10 02:11:49.330731 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.330650    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zccx2"]
Jul 10 02:11:49.333758 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.333739    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.336192 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.336168    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 10 02:11:49.336491 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.336474    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pk9p4\""
Jul 10 02:11:49.336615 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.336594    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 10 02:11:49.336791 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.336771    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 10 02:11:49.337412 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.337391    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 10 02:11:49.346874 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.346856    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zccx2"]
Jul 10 02:11:49.505938 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.505901    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.506106 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.505978    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0a41b0c6-0c4e-4605-b94f-3672543b59b3-crio-socket\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.506106 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.505999    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5jxd\" (UniqueName: \"kubernetes.io/projected/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-api-access-v5jxd\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.506106 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.506019    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a41b0c6-0c4e-4605-b94f-3672543b59b3-data-volume\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.506106 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.506039    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.606652 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.606560    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.606652 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.606646    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0a41b0c6-0c4e-4605-b94f-3672543b59b3-crio-socket\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.606776 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.606669    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5jxd\" (UniqueName: \"kubernetes.io/projected/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-api-access-v5jxd\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.606776 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.606688    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a41b0c6-0c4e-4605-b94f-3672543b59b3-data-volume\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.606776 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.606706    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.606776 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:49.606733    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:11:49.606931 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:49.606829    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls podName:0a41b0c6-0c4e-4605-b94f-3672543b59b3 nodeName:}" failed. No retries permitted until 2026-07-10 02:11:50.10679192 +0000 UTC m=+266.423101792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zccx2" (UID: "0a41b0c6-0c4e-4605-b94f-3672543b59b3") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:11:49.606931 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.606859    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0a41b0c6-0c4e-4605-b94f-3672543b59b3-crio-socket\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.607001 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:49.606929    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:11:49.607071 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:49.607058    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm podName:0a41b0c6-0c4e-4605-b94f-3672543b59b3 nodeName:}" failed. No retries permitted until 2026-07-10 02:11:50.107031032 +0000 UTC m=+266.423340904 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zccx2" (UID: "0a41b0c6-0c4e-4605-b94f-3672543b59b3") : configmap "kube-rbac-proxy" not found
Jul 10 02:11:49.607116 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.607067    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a41b0c6-0c4e-4605-b94f-3672543b59b3-data-volume\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:49.615584 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:49.615561    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5jxd\" (UniqueName: \"kubernetes.io/projected/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-api-access-v5jxd\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:50.110694 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:50.110664    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:50.110878 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:50.110713    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:50.110878 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:50.110778    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:11:50.110878 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:50.110796    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:11:50.110878 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:50.110857    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls podName:0a41b0c6-0c4e-4605-b94f-3672543b59b3 nodeName:}" failed. No retries permitted until 2026-07-10 02:11:51.110844892 +0000 UTC m=+267.427154743 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zccx2" (UID: "0a41b0c6-0c4e-4605-b94f-3672543b59b3") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:11:50.110878 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:50.110870    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm podName:0a41b0c6-0c4e-4605-b94f-3672543b59b3 nodeName:}" failed. No retries permitted until 2026-07-10 02:11:51.110864426 +0000 UTC m=+267.427174277 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zccx2" (UID: "0a41b0c6-0c4e-4605-b94f-3672543b59b3") : configmap "kube-rbac-proxy" not found
Jul 10 02:11:51.119541 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:51.119509    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:51.119925 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:51.119584    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:51.119925 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:51.119653    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:11:51.119925 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:51.119706    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls podName:0a41b0c6-0c4e-4605-b94f-3672543b59b3 nodeName:}" failed. No retries permitted until 2026-07-10 02:11:53.119689765 +0000 UTC m=+269.435999616 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zccx2" (UID: "0a41b0c6-0c4e-4605-b94f-3672543b59b3") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:11:51.119925 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:51.119656    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:11:51.119925 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:51.119772    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm podName:0a41b0c6-0c4e-4605-b94f-3672543b59b3 nodeName:}" failed. No retries permitted until 2026-07-10 02:11:53.119760719 +0000 UTC m=+269.436070569 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zccx2" (UID: "0a41b0c6-0c4e-4605-b94f-3672543b59b3") : configmap "kube-rbac-proxy" not found
Jul 10 02:11:51.221177 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:51.221149    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4gmwh_98184b4a-870b-47be-a682-e1f7d43effb0/dns-node-resolver/0.log"
Jul 10 02:11:52.621111 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:52.621084    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8w4fn_6e7c563a-d986-46de-994c-230d665e028b/node-ca/0.log"
Jul 10 02:11:53.134000 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:53.133964    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:53.134199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:53.134045    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:53.134199 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:53.134124    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:11:53.134199 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:53.134128    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:11:53.134199 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:53.134172    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm podName:0a41b0c6-0c4e-4605-b94f-3672543b59b3 nodeName:}" failed. No retries permitted until 2026-07-10 02:11:57.134159034 +0000 UTC m=+273.450468883 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zccx2" (UID: "0a41b0c6-0c4e-4605-b94f-3672543b59b3") : configmap "kube-rbac-proxy" not found
Jul 10 02:11:53.134199 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:53.134184    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls podName:0a41b0c6-0c4e-4605-b94f-3672543b59b3 nodeName:}" failed. No retries permitted until 2026-07-10 02:11:57.134178616 +0000 UTC m=+273.450488466 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zccx2" (UID: "0a41b0c6-0c4e-4605-b94f-3672543b59b3") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:11:53.422054 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:53.421968    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rv9wg_d86f5caf-4a63-4947-b175-ae4a035dd52b/serve-healthcheck-canary/0.log"
Jul 10 02:11:57.165185 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:57.165150    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:57.165717 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:11:57.165221    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:11:57.165717 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:57.165267    2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 10 02:11:57.165717 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:57.165346    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm podName:0a41b0c6-0c4e-4605-b94f-3672543b59b3 nodeName:}" failed. No retries permitted until 2026-07-10 02:12:05.165327409 +0000 UTC m=+281.481637263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zccx2" (UID: "0a41b0c6-0c4e-4605-b94f-3672543b59b3") : configmap "kube-rbac-proxy" not found
Jul 10 02:11:57.165717 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:57.165364    2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 10 02:11:57.165717 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:11:57.165435    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls podName:0a41b0c6-0c4e-4605-b94f-3672543b59b3 nodeName:}" failed. No retries permitted until 2026-07-10 02:12:05.165417937 +0000 UTC m=+281.481727803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zccx2" (UID: "0a41b0c6-0c4e-4605-b94f-3672543b59b3") : secret "insights-runtime-extractor-tls" not found
Jul 10 02:12:05.232647 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:05.232590    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:12:05.233156 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:05.232707    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:12:05.233232 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:05.233210    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0a41b0c6-0c4e-4605-b94f-3672543b59b3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:12:05.234927 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:05.234894    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0a41b0c6-0c4e-4605-b94f-3672543b59b3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zccx2\" (UID: \"0a41b0c6-0c4e-4605-b94f-3672543b59b3\") " pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:12:05.241906 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:05.241886    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zccx2"
Jul 10 02:12:05.363641 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:05.363605    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zccx2"]
Jul 10 02:12:05.367730 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:12:05.367696    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a41b0c6_0c4e_4605_b94f_3672543b59b3.slice/crio-7eccf638036d3ca9039d861ad0e89d27a7f30bada19dffa2d5dfa078213d9083 WatchSource:0}: Error finding container 7eccf638036d3ca9039d861ad0e89d27a7f30bada19dffa2d5dfa078213d9083: Status 404 returned error can't find the container with id 7eccf638036d3ca9039d861ad0e89d27a7f30bada19dffa2d5dfa078213d9083
Jul 10 02:12:06.152509 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:06.152397    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zccx2" event={"ID":"0a41b0c6-0c4e-4605-b94f-3672543b59b3","Type":"ContainerStarted","Data":"7eccf638036d3ca9039d861ad0e89d27a7f30bada19dffa2d5dfa078213d9083"}
Jul 10 02:12:09.802901 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:12:09.802859    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp" podUID="7008ee73-8f48-4845-89be-7c2a76aa76d7"
Jul 10 02:12:09.802901 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:12:09.802864    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6" podUID="109d498f-fabd-4b87-b893-19dc87cb1f23"
Jul 10 02:12:09.802901 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:12:09.802864    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vsgwk" podUID="d480e3b6-008b-433b-8a35-6d023ed58eff"
Jul 10 02:12:10.166909 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:10.166874    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zccx2" event={"ID":"0a41b0c6-0c4e-4605-b94f-3672543b59b3","Type":"ContainerStarted","Data":"5ac682629d5b1e38edb9d9698ca853a74e805d5aef61c4c2e997ac7220307a27"}
Jul 10 02:12:10.167069 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:10.166916    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:12:10.167069 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:10.166964    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zccx2" event={"ID":"0a41b0c6-0c4e-4605-b94f-3672543b59b3","Type":"ContainerStarted","Data":"347e27a7e626e1b4893acdaa14eb5521d34fb597b51e9a44595a31180ea895fc"}
Jul 10 02:12:10.167069 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:10.166880    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:12:10.167069 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:10.166880    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:12:13.097547 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.097504    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:12:13.097547 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.097543    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:12:13.098128 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.097574    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:12:13.100027 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.100001    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/109d498f-fabd-4b87-b893-19dc87cb1f23-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rr7f6\" (UID: \"109d498f-fabd-4b87-b893-19dc87cb1f23\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:12:13.100127 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.100044    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d480e3b6-008b-433b-8a35-6d023ed58eff-metrics-tls\") pod \"dns-default-vsgwk\" (UID: \"d480e3b6-008b-433b-8a35-6d023ed58eff\") " pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:12:13.100176 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.100152    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"image-registry-5f5cf64794-c75bp\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") " pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:12:13.170262 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.170227    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-58c2q\""
Jul 10 02:12:13.170262 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.170237    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g79n9\""
Jul 10 02:12:13.170449 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.170297    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-g8kf5\""
Jul 10 02:12:13.178176 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.178153    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"
Jul 10 02:12:13.178327 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.178184    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:12:13.178327 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.178206    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:12:13.179330 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.179309    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zccx2" event={"ID":"0a41b0c6-0c4e-4605-b94f-3672543b59b3","Type":"ContainerStarted","Data":"6b521c644c54984fc671a95c5e807da560130981b75da4e442378fd74c61e94f"}
Jul 10 02:12:13.198353 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.198295    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zccx2" podStartSLOduration=17.507146927 podStartE2EDuration="24.198271554s" totalImagesPullingTime="6.691124627s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:11:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:12:13.196435172 +0000 UTC m=+289.512745069" watchObservedRunningTime="2026-07-10 02:12:13.198271554 +0000 UTC m=+289.514581430"
Jul 10 02:12:13.320715 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.320529    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-rr7f6"]
Jul 10 02:12:13.323749 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:12:13.323723    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod109d498f_fabd_4b87_b893_19dc87cb1f23.slice/crio-b13dd388bd04af649f4dc45be7f3cacd39e322722cf3459d76479c1b075a7e91 WatchSource:0}: Error finding container b13dd388bd04af649f4dc45be7f3cacd39e322722cf3459d76479c1b075a7e91: Status 404 returned error can't find the container with id b13dd388bd04af649f4dc45be7f3cacd39e322722cf3459d76479c1b075a7e91
Jul 10 02:12:13.568175 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.568140    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vsgwk"]
Jul 10 02:12:13.570664 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:12:13.570636    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd480e3b6_008b_433b_8a35_6d023ed58eff.slice/crio-9a785712b0ab2bac49ec9c10a847336d91fade1274e3e5c48a29f60ae3b260a5 WatchSource:0}: Error finding container 9a785712b0ab2bac49ec9c10a847336d91fade1274e3e5c48a29f60ae3b260a5: Status 404 returned error can't find the container with id 9a785712b0ab2bac49ec9c10a847336d91fade1274e3e5c48a29f60ae3b260a5
Jul 10 02:12:13.574413 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:13.574381    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f5cf64794-c75bp"]
Jul 10 02:12:13.578283 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:12:13.578260    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7008ee73_8f48_4845_89be_7c2a76aa76d7.slice/crio-8a13f315f714bdb694977cba1ae9f6774303fb74c4d5094da7ed6d63776bdbc7 WatchSource:0}: Error finding container 8a13f315f714bdb694977cba1ae9f6774303fb74c4d5094da7ed6d63776bdbc7: Status 404 returned error can't find the container with id 8a13f315f714bdb694977cba1ae9f6774303fb74c4d5094da7ed6d63776bdbc7
Jul 10 02:12:14.184143 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:14.184104    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp" event={"ID":"7008ee73-8f48-4845-89be-7c2a76aa76d7","Type":"ContainerStarted","Data":"308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440"}
Jul 10 02:12:14.184622 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:14.184160    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp" event={"ID":"7008ee73-8f48-4845-89be-7c2a76aa76d7","Type":"ContainerStarted","Data":"8a13f315f714bdb694977cba1ae9f6774303fb74c4d5094da7ed6d63776bdbc7"}
Jul 10 02:12:14.184622 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:14.184185    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:12:14.185522 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:14.185491    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6" event={"ID":"109d498f-fabd-4b87-b893-19dc87cb1f23","Type":"ContainerStarted","Data":"b13dd388bd04af649f4dc45be7f3cacd39e322722cf3459d76479c1b075a7e91"}
Jul 10 02:12:14.186857 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:14.186828    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vsgwk" event={"ID":"d480e3b6-008b-433b-8a35-6d023ed58eff","Type":"ContainerStarted","Data":"9a785712b0ab2bac49ec9c10a847336d91fade1274e3e5c48a29f60ae3b260a5"}
Jul 10 02:12:14.204451 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:14.204384    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp" podStartSLOduration=289.20435768 podStartE2EDuration="4m49.20435768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:12:14.203293719 +0000 UTC m=+290.519603623" watchObservedRunningTime="2026-07-10 02:12:14.20435768 +0000 UTC m=+290.520667555"
Jul 10 02:12:15.191638 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:15.191600    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6" event={"ID":"109d498f-fabd-4b87-b893-19dc87cb1f23","Type":"ContainerStarted","Data":"a4a963ecc1f79daf2557f7cb520a8533b713f6d739a2d63310554bce85b433c9"}
Jul 10 02:12:15.209580 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:15.209528    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-rr7f6" podStartSLOduration=268.041701034 podStartE2EDuration="4m29.209513696s" totalImagesPullingTime="1.167812662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:12:15.208418135 +0000 UTC m=+291.524728007" watchObservedRunningTime="2026-07-10 02:12:15.209513696 +0000 UTC m=+291.525823567"
Jul 10 02:12:17.200897 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.200789    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vsgwk" event={"ID":"d480e3b6-008b-433b-8a35-6d023ed58eff","Type":"ContainerStarted","Data":"7d1100d739edc26e29fa8efefd0df62f03f4e5cf1f8e9909022354f240dd2013"}
Jul 10 02:12:17.201306 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.200957    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:12:17.201306 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.200995    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vsgwk" event={"ID":"d480e3b6-008b-433b-8a35-6d023ed58eff","Type":"ContainerStarted","Data":"f2e2b559a694a46ec2a83f2590a9d581d8cc8ca7ec3d916892fb0cf80226ce3e"}
Jul 10 02:12:17.202531 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.202508    2557 generic.go:356] "Generic (PLEG): container finished" podID="c12cbd8d-4e9c-44cb-a769-f34924a81c46" containerID="277c63b7480508564b4dc5e1b7b80ef02bf901e3f90f0109414f97938f29eef4" exitCode=255
Jul 10 02:12:17.202632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.202582    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx" event={"ID":"c12cbd8d-4e9c-44cb-a769-f34924a81c46","Type":"ContainerDied","Data":"277c63b7480508564b4dc5e1b7b80ef02bf901e3f90f0109414f97938f29eef4"}
Jul 10 02:12:17.202632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.202620    2557 scope.go:117] "RemoveContainer" containerID="ebae43471c61a90c99f3986b8b71b27ec8738cc3f86684b4c54c494ba98ded30"
Jul 10 02:12:17.202921 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.202903    2557 scope.go:117] "RemoveContainer" containerID="277c63b7480508564b4dc5e1b7b80ef02bf901e3f90f0109414f97938f29eef4"
Jul 10 02:12:17.203070 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:12:17.203049    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx_open-cluster-management-agent-addon(c12cbd8d-4e9c-44cb-a769-f34924a81c46)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx" podUID="c12cbd8d-4e9c-44cb-a769-f34924a81c46"
Jul 10 02:12:17.204281 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.204261    2557 generic.go:356] "Generic (PLEG): container finished" podID="0444260c-a7df-4444-960a-c077b5a09f8f" containerID="2710a1ea515af5ca39aa8ce45bc4505917e9db4fba11d6f6cb1241631763ebf8" exitCode=1
Jul 10 02:12:17.204365 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.204336    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl" event={"ID":"0444260c-a7df-4444-960a-c077b5a09f8f","Type":"ContainerDied","Data":"2710a1ea515af5ca39aa8ce45bc4505917e9db4fba11d6f6cb1241631763ebf8"}
Jul 10 02:12:17.204583 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.204570    2557 scope.go:117] "RemoveContainer" containerID="2710a1ea515af5ca39aa8ce45bc4505917e9db4fba11d6f6cb1241631763ebf8"
Jul 10 02:12:17.204711 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:12:17.204695    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-f944b9f5-mmttl_open-cluster-management-agent-addon(0444260c-a7df-4444-960a-c077b5a09f8f)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl" podUID="0444260c-a7df-4444-960a-c077b5a09f8f"
Jul 10 02:12:17.213301 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.213283    2557 scope.go:117] "RemoveContainer" containerID="19b48ec41be07947c086dbbe81a1a7ce855c98254dc6122a5eb67e8b74d0f8c2"
Jul 10 02:12:17.218644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.218599    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vsgwk" podStartSLOduration=251.292593158 podStartE2EDuration="4m14.218586378s" totalImagesPullingTime="2.92599322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:08:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:12:17.217191091 +0000 UTC m=+293.533500983" watchObservedRunningTime="2026-07-10 02:12:17.218586378 +0000 UTC m=+293.534896228"
Jul 10 02:12:17.844511 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:17.844475    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:12:18.209893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:18.209819    2557 scope.go:117] "RemoveContainer" containerID="2710a1ea515af5ca39aa8ce45bc4505917e9db4fba11d6f6cb1241631763ebf8"
Jul 10 02:12:18.210238 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:12:18.209977    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-f944b9f5-mmttl_open-cluster-management-agent-addon(0444260c-a7df-4444-960a-c077b5a09f8f)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl" podUID="0444260c-a7df-4444-960a-c077b5a09f8f"
Jul 10 02:12:19.754149 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:19.754105    2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:12:19.754529 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:19.754499    2557 scope.go:117] "RemoveContainer" containerID="2710a1ea515af5ca39aa8ce45bc4505917e9db4fba11d6f6cb1241631763ebf8"
Jul 10 02:12:19.754681 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:12:19.754647    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-f944b9f5-mmttl_open-cluster-management-agent-addon(0444260c-a7df-4444-960a-c077b5a09f8f)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl" podUID="0444260c-a7df-4444-960a-c077b5a09f8f"
Jul 10 02:12:19.772899 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:19.772867    2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx"
Jul 10 02:12:19.773188 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:19.773176    2557 scope.go:117] "RemoveContainer" containerID="277c63b7480508564b4dc5e1b7b80ef02bf901e3f90f0109414f97938f29eef4"
Jul 10 02:12:19.773330 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:12:19.773314    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx_open-cluster-management-agent-addon(c12cbd8d-4e9c-44cb-a769-f34924a81c46)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx" podUID="c12cbd8d-4e9c-44cb-a769-f34924a81c46"
Jul 10 02:12:24.101983 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:24.101957    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:12:24.103518 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:24.103494    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:12:24.107667 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:24.107651    2557 kubelet.go:1648] "Image garbage collection succeeded"
Jul 10 02:12:24.237291 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:12:24.237143    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5j8nl" podUID="1a790021-08d0-4e49-8da8-f046861c72ac"
Jul 10 02:12:26.125364 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.125336    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-765lk"]
Jul 10 02:12:26.130367 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.130346    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.131906 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.131876    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 10 02:12:26.132021 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.131985    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 10 02:12:26.132262 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.132246    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 10 02:12:26.132323 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.132247    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 10 02:12:26.132323 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.132303    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 10 02:12:26.132410 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.132341    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 10 02:12:26.132587 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.132574    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hg988\""
Jul 10 02:12:26.194280 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.194244    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.194466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.194302    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a4e5f18-4506-4200-ad5b-44b0d14b7337-root\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.194466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.194357    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjd9d\" (UniqueName: \"kubernetes.io/projected/5a4e5f18-4506-4200-ad5b-44b0d14b7337-kube-api-access-gjd9d\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.194466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.194378    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-tls\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.194466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.194440    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a4e5f18-4506-4200-ad5b-44b0d14b7337-metrics-client-ca\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.194641 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.194501    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-accelerators-collector-config\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.194641 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.194528    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a4e5f18-4506-4200-ad5b-44b0d14b7337-sys\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.194641 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.194546    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-wtmp\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.194641 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.194568    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-textfile\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.294953 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.294919    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295127 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.294965    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a4e5f18-4506-4200-ad5b-44b0d14b7337-root\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295127 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.294985    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjd9d\" (UniqueName: \"kubernetes.io/projected/5a4e5f18-4506-4200-ad5b-44b0d14b7337-kube-api-access-gjd9d\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295127 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295002    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-tls\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295127 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295018    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a4e5f18-4506-4200-ad5b-44b0d14b7337-metrics-client-ca\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295127 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295038    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-accelerators-collector-config\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295127 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295036    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5a4e5f18-4506-4200-ad5b-44b0d14b7337-root\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295127 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295062    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a4e5f18-4506-4200-ad5b-44b0d14b7337-sys\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295127 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295118    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5a4e5f18-4506-4200-ad5b-44b0d14b7337-sys\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295472 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:12:26.295136    2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 10 02:12:26.295472 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295176    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-wtmp\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295472 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:12:26.295212    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-tls podName:5a4e5f18-4506-4200-ad5b-44b0d14b7337 nodeName:}" failed. No retries permitted until 2026-07-10 02:12:26.795190677 +0000 UTC m=+303.111500548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-tls") pod "node-exporter-765lk" (UID: "5a4e5f18-4506-4200-ad5b-44b0d14b7337") : secret "node-exporter-tls" not found
Jul 10 02:12:26.295472 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295233    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-textfile\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295472 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295303    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-wtmp\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295598    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-textfile\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295618    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-accelerators-collector-config\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.295688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.295627    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a4e5f18-4506-4200-ad5b-44b0d14b7337-metrics-client-ca\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.297260 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.297243    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.302987 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.302963    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjd9d\" (UniqueName: \"kubernetes.io/projected/5a4e5f18-4506-4200-ad5b-44b0d14b7337-kube-api-access-gjd9d\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.797661 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.797626    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-tls\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:26.799833 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:26.799792    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5a4e5f18-4506-4200-ad5b-44b0d14b7337-node-exporter-tls\") pod \"node-exporter-765lk\" (UID: \"5a4e5f18-4506-4200-ad5b-44b0d14b7337\") " pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:27.039280 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:27.039240    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-765lk"
Jul 10 02:12:27.050564 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:12:27.050501    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a4e5f18_4506_4200_ad5b_44b0d14b7337.slice/crio-f862b4473b97e528ffbbe3a77ad1a35bc04dd9f4598bf6e9b7d059bd0aafa54a WatchSource:0}: Error finding container f862b4473b97e528ffbbe3a77ad1a35bc04dd9f4598bf6e9b7d059bd0aafa54a: Status 404 returned error can't find the container with id f862b4473b97e528ffbbe3a77ad1a35bc04dd9f4598bf6e9b7d059bd0aafa54a
Jul 10 02:12:27.052894 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:27.052874    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:12:27.211902 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:27.211873    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vsgwk"
Jul 10 02:12:27.238524 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:27.238494    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-765lk" event={"ID":"5a4e5f18-4506-4200-ad5b-44b0d14b7337","Type":"ContainerStarted","Data":"f862b4473b97e528ffbbe3a77ad1a35bc04dd9f4598bf6e9b7d059bd0aafa54a"}
Jul 10 02:12:29.246139 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:29.246096    2557 generic.go:356] "Generic (PLEG): container finished" podID="5a4e5f18-4506-4200-ad5b-44b0d14b7337" containerID="c8064e7ec74a314d5daa7f3cc9bdaf70742fa529a7b218d6fc7c9d4efbb81ec2" exitCode=0
Jul 10 02:12:29.246515 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:29.246153    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-765lk" event={"ID":"5a4e5f18-4506-4200-ad5b-44b0d14b7337","Type":"ContainerDied","Data":"c8064e7ec74a314d5daa7f3cc9bdaf70742fa529a7b218d6fc7c9d4efbb81ec2"}
Jul 10 02:12:30.251211 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:30.251175    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-765lk" event={"ID":"5a4e5f18-4506-4200-ad5b-44b0d14b7337","Type":"ContainerStarted","Data":"0ddbd19e0824745713862736963e292a389f4ef0111ca3527ad2ea84c55da405"}
Jul 10 02:12:30.251716 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:30.251227    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-765lk" event={"ID":"5a4e5f18-4506-4200-ad5b-44b0d14b7337","Type":"ContainerStarted","Data":"c4e491a94930110de67a226360399601efc1854e068a85b9f31d7c98933cd220"}
Jul 10 02:12:30.269490 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:30.269443    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-765lk" podStartSLOduration=2.996537162 podStartE2EDuration="4.269428707s" totalImagesPullingTime="1.210209678s" totalInitContainerRuntime="62.681867ms" isStatefulPod=true podCreationTimestamp="2026-07-10 02:12:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:12:30.26851966 +0000 UTC m=+306.584829544" watchObservedRunningTime="2026-07-10 02:12:30.269428707 +0000 UTC m=+306.585738578"
Jul 10 02:12:33.183287 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:33.183254    2557 patch_prober.go:28] interesting pod/image-registry-5f5cf64794-c75bp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 10 02:12:33.183644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:33.183334    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp" podUID="7008ee73-8f48-4845-89be-7c2a76aa76d7" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:12:34.239012 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:34.238981    2557 scope.go:117] "RemoveContainer" containerID="277c63b7480508564b4dc5e1b7b80ef02bf901e3f90f0109414f97938f29eef4"
Jul 10 02:12:34.480037 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:34.479998    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f5cf64794-c75bp"]
Jul 10 02:12:34.484597 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:34.484575    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:12:35.236721 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:35.236689    2557 scope.go:117] "RemoveContainer" containerID="2710a1ea515af5ca39aa8ce45bc4505917e9db4fba11d6f6cb1241631763ebf8"
Jul 10 02:12:35.268627 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:35.268599    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5895b8fc5b-mkxqx" event={"ID":"c12cbd8d-4e9c-44cb-a769-f34924a81c46","Type":"ContainerStarted","Data":"aa34c9c5660e3bfcf6e215e7d93ce75303aa286990e9f83782d9f8162dedcd2c"}
Jul 10 02:12:36.273218 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:36.273184    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl" event={"ID":"0444260c-a7df-4444-960a-c077b5a09f8f","Type":"ContainerStarted","Data":"28ead8667cc2a1aa1612296e21da7438c364b755d6165b6c43b997c39885bc91"}
Jul 10 02:12:36.273662 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:36.273465    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:12:36.274941 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:36.274920    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-f944b9f5-mmttl"
Jul 10 02:12:39.236606 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:39.236574    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:12:59.499791 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.499725    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp" podUID="7008ee73-8f48-4845-89be-7c2a76aa76d7" containerName="registry" containerID="cri-o://308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440" gracePeriod=30
Jul 10 02:12:59.738606 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.738581    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:12:59.859955 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.859737    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-image-registry-private-configuration\") pod \"7008ee73-8f48-4845-89be-7c2a76aa76d7\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") "
Jul 10 02:12:59.860155 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.860122    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd2ph\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-kube-api-access-dd2ph\") pod \"7008ee73-8f48-4845-89be-7c2a76aa76d7\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") "
Jul 10 02:12:59.860218 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.860186    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") pod \"7008ee73-8f48-4845-89be-7c2a76aa76d7\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") "
Jul 10 02:12:59.860218 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.860212    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-certificates\") pod \"7008ee73-8f48-4845-89be-7c2a76aa76d7\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") "
Jul 10 02:12:59.860336 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.860310    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-trusted-ca\") pod \"7008ee73-8f48-4845-89be-7c2a76aa76d7\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") "
Jul 10 02:12:59.860395 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.860382    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-bound-sa-token\") pod \"7008ee73-8f48-4845-89be-7c2a76aa76d7\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") "
Jul 10 02:12:59.860449 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.860420    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7008ee73-8f48-4845-89be-7c2a76aa76d7-ca-trust-extracted\") pod \"7008ee73-8f48-4845-89be-7c2a76aa76d7\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") "
Jul 10 02:12:59.860498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.860453    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-installation-pull-secrets\") pod \"7008ee73-8f48-4845-89be-7c2a76aa76d7\" (UID: \"7008ee73-8f48-4845-89be-7c2a76aa76d7\") "
Jul 10 02:12:59.862024 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.861988    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7008ee73-8f48-4845-89be-7c2a76aa76d7" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:12:59.862589 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.862564    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-trusted-ca\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:12:59.862956 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.862928    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7008ee73-8f48-4845-89be-7c2a76aa76d7" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:12:59.863254 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.863126    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7008ee73-8f48-4845-89be-7c2a76aa76d7" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:12:59.863856 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.863402    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7008ee73-8f48-4845-89be-7c2a76aa76d7" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:12:59.863856 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.863435    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-kube-api-access-dd2ph" (OuterVolumeSpecName: "kube-api-access-dd2ph") pod "7008ee73-8f48-4845-89be-7c2a76aa76d7" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7"). InnerVolumeSpecName "kube-api-access-dd2ph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:12:59.867361 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.867333    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7008ee73-8f48-4845-89be-7c2a76aa76d7" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:12:59.867964 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.867943    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7008ee73-8f48-4845-89be-7c2a76aa76d7" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:12:59.872103 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.872077    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7008ee73-8f48-4845-89be-7c2a76aa76d7-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7008ee73-8f48-4845-89be-7c2a76aa76d7" (UID: "7008ee73-8f48-4845-89be-7c2a76aa76d7"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:12:59.963580 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.963530    2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-tls\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:12:59.963580 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.963574    2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7008ee73-8f48-4845-89be-7c2a76aa76d7-registry-certificates\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:12:59.963580 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.963588    2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-bound-sa-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:12:59.963580 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.963596    2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7008ee73-8f48-4845-89be-7c2a76aa76d7-ca-trust-extracted\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:12:59.963872 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.963604    2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-installation-pull-secrets\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:12:59.963872 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.963614    2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7008ee73-8f48-4845-89be-7c2a76aa76d7-image-registry-private-configuration\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:12:59.963872 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:12:59.963622    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd2ph\" (UniqueName: \"kubernetes.io/projected/7008ee73-8f48-4845-89be-7c2a76aa76d7-kube-api-access-dd2ph\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:13:00.348942 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:00.348902    2557 generic.go:356] "Generic (PLEG): container finished" podID="7008ee73-8f48-4845-89be-7c2a76aa76d7" containerID="308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440" exitCode=0
Jul 10 02:13:00.349107 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:00.348957    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp"
Jul 10 02:13:00.349107 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:00.348969    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp" event={"ID":"7008ee73-8f48-4845-89be-7c2a76aa76d7","Type":"ContainerDied","Data":"308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440"}
Jul 10 02:13:00.349107 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:00.349012    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f5cf64794-c75bp" event={"ID":"7008ee73-8f48-4845-89be-7c2a76aa76d7","Type":"ContainerDied","Data":"8a13f315f714bdb694977cba1ae9f6774303fb74c4d5094da7ed6d63776bdbc7"}
Jul 10 02:13:00.349107 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:00.349032    2557 scope.go:117] "RemoveContainer" containerID="308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440"
Jul 10 02:13:00.356745 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:00.356716    2557 scope.go:117] "RemoveContainer" containerID="308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440"
Jul 10 02:13:00.357054 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:13:00.357032    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440\": container with ID starting with 308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440 not found: ID does not exist" containerID="308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440"
Jul 10 02:13:00.357111 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:00.357072    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440"} err="failed to get container status \"308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440\": rpc error: code = NotFound desc = could not find container \"308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440\": container with ID starting with 308a3656dd42fc70ac2821b8e98249f5377a16c11575240a3b45300613b67440 not found: ID does not exist"
Jul 10 02:13:00.366748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:00.366723    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f5cf64794-c75bp"]
Jul 10 02:13:00.368989 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:00.368969    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f5cf64794-c75bp"]
Jul 10 02:13:02.240897 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:02.240862    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7008ee73-8f48-4845-89be-7c2a76aa76d7" path="/var/lib/kubelet/pods/7008ee73-8f48-4845-89be-7c2a76aa76d7/volumes"
Jul 10 02:13:19.789458 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:19.789375    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" podUID="df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 10 02:13:29.789202 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:29.789161    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" podUID="df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 10 02:13:37.216632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:37.216571    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:13:37.218863 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:37.218842    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1a790021-08d0-4e49-8da8-f046861c72ac-metrics-certs\") pod \"network-metrics-daemon-5j8nl\" (UID: \"1a790021-08d0-4e49-8da8-f046861c72ac\") " pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:13:37.439931 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:37.439898    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dzm46\""
Jul 10 02:13:37.448392 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:37.448373    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5j8nl"
Jul 10 02:13:37.572056 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:37.572025    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5j8nl"]
Jul 10 02:13:37.574952 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:13:37.574928    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a790021_08d0_4e49_8da8_f046861c72ac.slice/crio-90cb0809ef0f365ab8683305ab9be56bdbb8e20fffe6e0ffc4cd3bf6b8d58553 WatchSource:0}: Error finding container 90cb0809ef0f365ab8683305ab9be56bdbb8e20fffe6e0ffc4cd3bf6b8d58553: Status 404 returned error can't find the container with id 90cb0809ef0f365ab8683305ab9be56bdbb8e20fffe6e0ffc4cd3bf6b8d58553
Jul 10 02:13:38.469529 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:38.469485    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5j8nl" event={"ID":"1a790021-08d0-4e49-8da8-f046861c72ac","Type":"ContainerStarted","Data":"90cb0809ef0f365ab8683305ab9be56bdbb8e20fffe6e0ffc4cd3bf6b8d58553"}
Jul 10 02:13:39.474532 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:39.474492    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5j8nl" event={"ID":"1a790021-08d0-4e49-8da8-f046861c72ac","Type":"ContainerStarted","Data":"1200c015c23bb54751883d045393a3f3498b153217b9888dc47659f7bd82fc4f"}
Jul 10 02:13:39.474934 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:39.474553    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5j8nl" event={"ID":"1a790021-08d0-4e49-8da8-f046861c72ac","Type":"ContainerStarted","Data":"b3982cf12b47764076757bec8a1bb0eac554893ec5a82aaf7c803409ca6fef17"}
Jul 10 02:13:39.491346 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:39.491299    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5j8nl" podStartSLOduration=373.230084479 podStartE2EDuration="6m14.491283353s" totalImagesPullingTime="1.261198874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:07:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:13:39.489827513 +0000 UTC m=+375.806137387" watchObservedRunningTime="2026-07-10 02:13:39.491283353 +0000 UTC m=+375.807593222"
Jul 10 02:13:39.790096 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:39.790060    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" podUID="df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 10 02:13:39.790256 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:39.790142    2557 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt"
Jul 10 02:13:39.790605 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:39.790585    2557 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"c765594265f9344a5297653c7231ab4240a8dcbfdb3581ebad5bab41a5927be8"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 10 02:13:39.790695 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:39.790622    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" podUID="df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb" containerName="service-proxy" containerID="cri-o://c765594265f9344a5297653c7231ab4240a8dcbfdb3581ebad5bab41a5927be8" gracePeriod=30
Jul 10 02:13:40.480322 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:40.480289    2557 generic.go:356] "Generic (PLEG): container finished" podID="df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb" containerID="c765594265f9344a5297653c7231ab4240a8dcbfdb3581ebad5bab41a5927be8" exitCode=2
Jul 10 02:13:40.480731 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:40.480366    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" event={"ID":"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb","Type":"ContainerDied","Data":"c765594265f9344a5297653c7231ab4240a8dcbfdb3581ebad5bab41a5927be8"}
Jul 10 02:13:40.480731 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:13:40.480410    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-86ccb77c9d-dx6mt" event={"ID":"df4d97b8-bff0-4ab2-8aa7-3ad14dd5efbb","Type":"ContainerStarted","Data":"41a59fc9df8274605111c0d6ad19ab5cb3a95746ebb94190955d3c6342127eba"}
Jul 10 02:15:29.246822 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.246782    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p"]
Jul 10 02:15:29.247303 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.247018    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7008ee73-8f48-4845-89be-7c2a76aa76d7" containerName="registry"
Jul 10 02:15:29.247303 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.247028    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7008ee73-8f48-4845-89be-7c2a76aa76d7" containerName="registry"
Jul 10 02:15:29.247303 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.247070    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7008ee73-8f48-4845-89be-7c2a76aa76d7" containerName="registry"
Jul 10 02:15:29.249615 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.249595    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p"
Jul 10 02:15:29.251222 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.251199    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 10 02:15:29.251353 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.251317    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-pjdql\""
Jul 10 02:15:29.251586 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.251570    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 10 02:15:29.260492 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.260472    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p"]
Jul 10 02:15:29.286236 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.286207    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f28baf9-70d7-4078-8fa4-6d7dd881321b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2dn8p\" (UID: \"6f28baf9-70d7-4078-8fa4-6d7dd881321b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p"
Jul 10 02:15:29.286236 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.286239    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhwsc\" (UniqueName: \"kubernetes.io/projected/6f28baf9-70d7-4078-8fa4-6d7dd881321b-kube-api-access-rhwsc\") pod \"openshift-lws-operator-fd8ccff4c-2dn8p\" (UID: \"6f28baf9-70d7-4078-8fa4-6d7dd881321b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p"
Jul 10 02:15:29.387466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.387435    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhwsc\" (UniqueName: \"kubernetes.io/projected/6f28baf9-70d7-4078-8fa4-6d7dd881321b-kube-api-access-rhwsc\") pod \"openshift-lws-operator-fd8ccff4c-2dn8p\" (UID: \"6f28baf9-70d7-4078-8fa4-6d7dd881321b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p"
Jul 10 02:15:29.387623 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.387500    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f28baf9-70d7-4078-8fa4-6d7dd881321b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2dn8p\" (UID: \"6f28baf9-70d7-4078-8fa4-6d7dd881321b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p"
Jul 10 02:15:29.387795 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.387779    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6f28baf9-70d7-4078-8fa4-6d7dd881321b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2dn8p\" (UID: \"6f28baf9-70d7-4078-8fa4-6d7dd881321b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p"
Jul 10 02:15:29.405132 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.405099    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhwsc\" (UniqueName: \"kubernetes.io/projected/6f28baf9-70d7-4078-8fa4-6d7dd881321b-kube-api-access-rhwsc\") pod \"openshift-lws-operator-fd8ccff4c-2dn8p\" (UID: \"6f28baf9-70d7-4078-8fa4-6d7dd881321b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p"
Jul 10 02:15:29.557869 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.557845    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p"
Jul 10 02:15:29.680837 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.680791    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p"]
Jul 10 02:15:29.683504 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:15:29.683478    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f28baf9_70d7_4078_8fa4_6d7dd881321b.slice/crio-a34740522966dc0ebc2f5329291abf35f78257e03e624cfc7217364a0532d530 WatchSource:0}: Error finding container a34740522966dc0ebc2f5329291abf35f78257e03e624cfc7217364a0532d530: Status 404 returned error can't find the container with id a34740522966dc0ebc2f5329291abf35f78257e03e624cfc7217364a0532d530
Jul 10 02:15:29.837059 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:29.836989    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p" event={"ID":"6f28baf9-70d7-4078-8fa4-6d7dd881321b","Type":"ContainerStarted","Data":"a34740522966dc0ebc2f5329291abf35f78257e03e624cfc7217364a0532d530"}
Jul 10 02:15:47.898459 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:47.898414    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p" event={"ID":"6f28baf9-70d7-4078-8fa4-6d7dd881321b","Type":"ContainerStarted","Data":"18f51d9692cce2a04bd112b5c7053157d5449ea7758d0ce9c49c7e5d4fab2eef"}
Jul 10 02:15:47.913766 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:15:47.913718    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2dn8p" podStartSLOduration=1.638093386 podStartE2EDuration="18.913703541s" totalImagesPullingTime="17.275610155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:15:47.913243491 +0000 UTC m=+504.229553363" watchObservedRunningTime="2026-07-10 02:15:47.913703541 +0000 UTC m=+504.230013413"
Jul 10 02:16:09.195746 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.195696    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"]
Jul 10 02:16:09.199107 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.199080    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.201218 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.201199    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 10 02:16:09.201315 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.201267    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rnfzq\""
Jul 10 02:16:09.201752 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.201733    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 10 02:16:09.201790 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.201778    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 10 02:16:09.202826 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.202788    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 10 02:16:09.214968 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.214948    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"]
Jul 10 02:16:09.270308 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.270276    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-manifests\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.270468 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.270312    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9t4w\" (UniqueName: \"kubernetes.io/projected/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-kube-api-access-m9t4w\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.270468 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.270340    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.270468 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.270358    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-webhook-cert\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.270468 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.270401    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-tmp\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.371344 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.371302    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-tmp\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.371539 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.371454    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-manifests\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.371539 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.371495    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9t4w\" (UniqueName: \"kubernetes.io/projected/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-kube-api-access-m9t4w\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.371539 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.371531    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.371705 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.371555    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-webhook-cert\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.371886 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.371861    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-manifests\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.373650 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.373620    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-tmp\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.373888 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.373868    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-webhook-cert\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.374011 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.373992    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.381571 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.381541    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9t4w\" (UniqueName: \"kubernetes.io/projected/d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9-kube-api-access-m9t4w\") pod \"opendatahub-operator-controller-manager-78b9cbb96-6f4c6\" (UID: \"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9\") " pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.509710 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.509616    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:09.643631 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.643597    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"]
Jul 10 02:16:09.646348 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:16:09.646322    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70fd7b3_cbbb_4a89_8a58_576aa6ae40b9.slice/crio-3c178ea19132578b5da352d04aed2a692f855c5dcd1d1d83e4358204d0bbb468 WatchSource:0}: Error finding container 3c178ea19132578b5da352d04aed2a692f855c5dcd1d1d83e4358204d0bbb468: Status 404 returned error can't find the container with id 3c178ea19132578b5da352d04aed2a692f855c5dcd1d1d83e4358204d0bbb468
Jul 10 02:16:09.975860 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:09.975796    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6" event={"ID":"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9","Type":"ContainerStarted","Data":"3c178ea19132578b5da352d04aed2a692f855c5dcd1d1d83e4358204d0bbb468"}
Jul 10 02:16:12.988305 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:12.988216    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6" event={"ID":"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9","Type":"ContainerStarted","Data":"dbbf4a7d58b3d3ba49f778bb2c2f77c626294572590dc2413181160f40a105e7"}
Jul 10 02:16:13.992573 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:13.992489    2557 generic.go:356] "Generic (PLEG): container finished" podID="d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9" containerID="dbbf4a7d58b3d3ba49f778bb2c2f77c626294572590dc2413181160f40a105e7" exitCode=0
Jul 10 02:16:13.992946 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:13.992573    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6" event={"ID":"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9","Type":"ContainerDied","Data":"dbbf4a7d58b3d3ba49f778bb2c2f77c626294572590dc2413181160f40a105e7"}
Jul 10 02:16:14.997573 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:14.997537    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6" event={"ID":"d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9","Type":"ContainerStarted","Data":"16ff64a05c1be1bf22fe57242578e556e497d3aba4aa73073db1c68c820cbd0a"}
Jul 10 02:16:14.997970 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:14.997631    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:15.018310 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:15.018232    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6" podStartSLOduration=1.589074583 podStartE2EDuration="6.018215214s" totalImagesPullingTime="3.409606337s" totalInitContainerRuntime="1.019534294s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:16:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:15.017516517 +0000 UTC m=+531.333826400" watchObservedRunningTime="2026-07-10 02:16:15.018215214 +0000 UTC m=+531.334525087"
Jul 10 02:16:26.003860 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:26.003764    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78b9cbb96-6f4c6"
Jul 10 02:16:28.819217 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.819187    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"]
Jul 10 02:16:28.824973 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.824951    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"
Jul 10 02:16:28.826929 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.826893    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 10 02:16:28.826929 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.826916    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 10 02:16:28.827085 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.826984    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 10 02:16:28.827085 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.826991    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 10 02:16:28.827232 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.827219    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9gm8x\""
Jul 10 02:16:28.830493 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.830473    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"]
Jul 10 02:16:28.917575 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.917531    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2cf43624-9057-462b-ba5e-e3a35ae43ec6-tmp\") pod \"kube-auth-proxy-6cf46c8b8c-7gbvk\" (UID: \"2cf43624-9057-462b-ba5e-e3a35ae43ec6\") " pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"
Jul 10 02:16:28.917743 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.917582    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p89m\" (UniqueName: \"kubernetes.io/projected/2cf43624-9057-462b-ba5e-e3a35ae43ec6-kube-api-access-7p89m\") pod \"kube-auth-proxy-6cf46c8b8c-7gbvk\" (UID: \"2cf43624-9057-462b-ba5e-e3a35ae43ec6\") " pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"
Jul 10 02:16:28.917743 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.917665    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cf43624-9057-462b-ba5e-e3a35ae43ec6-tls-certs\") pod \"kube-auth-proxy-6cf46c8b8c-7gbvk\" (UID: \"2cf43624-9057-462b-ba5e-e3a35ae43ec6\") " pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"
Jul 10 02:16:28.943488 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.943463    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2b76z"]
Jul 10 02:16:28.946215 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.946198    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2b76z"
Jul 10 02:16:28.948919 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.948898    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 10 02:16:28.949022 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.949005    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-jmfbb\""
Jul 10 02:16:28.963990 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:28.963964    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2b76z"]
Jul 10 02:16:29.018157 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.018125    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2cf43624-9057-462b-ba5e-e3a35ae43ec6-tmp\") pod \"kube-auth-proxy-6cf46c8b8c-7gbvk\" (UID: \"2cf43624-9057-462b-ba5e-e3a35ae43ec6\") " pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"
Jul 10 02:16:29.018303 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.018170    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p89m\" (UniqueName: \"kubernetes.io/projected/2cf43624-9057-462b-ba5e-e3a35ae43ec6-kube-api-access-7p89m\") pod \"kube-auth-proxy-6cf46c8b8c-7gbvk\" (UID: \"2cf43624-9057-462b-ba5e-e3a35ae43ec6\") " pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"
Jul 10 02:16:29.018303 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.018208    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cf43624-9057-462b-ba5e-e3a35ae43ec6-tls-certs\") pod \"kube-auth-proxy-6cf46c8b8c-7gbvk\" (UID: \"2cf43624-9057-462b-ba5e-e3a35ae43ec6\") " pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"
Jul 10 02:16:29.020290 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.020261    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2cf43624-9057-462b-ba5e-e3a35ae43ec6-tmp\") pod \"kube-auth-proxy-6cf46c8b8c-7gbvk\" (UID: \"2cf43624-9057-462b-ba5e-e3a35ae43ec6\") " pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"
Jul 10 02:16:29.020492 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.020474    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cf43624-9057-462b-ba5e-e3a35ae43ec6-tls-certs\") pod \"kube-auth-proxy-6cf46c8b8c-7gbvk\" (UID: \"2cf43624-9057-462b-ba5e-e3a35ae43ec6\") " pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"
Jul 10 02:16:29.028249 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.028217    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p89m\" (UniqueName: \"kubernetes.io/projected/2cf43624-9057-462b-ba5e-e3a35ae43ec6-kube-api-access-7p89m\") pod \"kube-auth-proxy-6cf46c8b8c-7gbvk\" (UID: \"2cf43624-9057-462b-ba5e-e3a35ae43ec6\") " pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"
Jul 10 02:16:29.119194 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.119093    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4af3523-66b5-4c1e-a6e9-4292aeb12df0-cert\") pod \"odh-model-controller-7998c67dc4-2b76z\" (UID: \"c4af3523-66b5-4c1e-a6e9-4292aeb12df0\") " pod="opendatahub/odh-model-controller-7998c67dc4-2b76z"
Jul 10 02:16:29.119194 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.119163    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjpzx\" (UniqueName: \"kubernetes.io/projected/c4af3523-66b5-4c1e-a6e9-4292aeb12df0-kube-api-access-jjpzx\") pod \"odh-model-controller-7998c67dc4-2b76z\" (UID: \"c4af3523-66b5-4c1e-a6e9-4292aeb12df0\") " pod="opendatahub/odh-model-controller-7998c67dc4-2b76z"
Jul 10 02:16:29.135093 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.135065    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"
Jul 10 02:16:29.220263 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.220231    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjpzx\" (UniqueName: \"kubernetes.io/projected/c4af3523-66b5-4c1e-a6e9-4292aeb12df0-kube-api-access-jjpzx\") pod \"odh-model-controller-7998c67dc4-2b76z\" (UID: \"c4af3523-66b5-4c1e-a6e9-4292aeb12df0\") " pod="opendatahub/odh-model-controller-7998c67dc4-2b76z"
Jul 10 02:16:29.220401 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.220309    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4af3523-66b5-4c1e-a6e9-4292aeb12df0-cert\") pod \"odh-model-controller-7998c67dc4-2b76z\" (UID: \"c4af3523-66b5-4c1e-a6e9-4292aeb12df0\") " pod="opendatahub/odh-model-controller-7998c67dc4-2b76z"
Jul 10 02:16:29.220456 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:29.220418    2557 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 10 02:16:29.220488 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:29.220478    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4af3523-66b5-4c1e-a6e9-4292aeb12df0-cert podName:c4af3523-66b5-4c1e-a6e9-4292aeb12df0 nodeName:}" failed. No retries permitted until 2026-07-10 02:16:29.720459077 +0000 UTC m=+546.036768939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c4af3523-66b5-4c1e-a6e9-4292aeb12df0-cert") pod "odh-model-controller-7998c67dc4-2b76z" (UID: "c4af3523-66b5-4c1e-a6e9-4292aeb12df0") : secret "odh-model-controller-webhook-cert" not found
Jul 10 02:16:29.231732 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.231699    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjpzx\" (UniqueName: \"kubernetes.io/projected/c4af3523-66b5-4c1e-a6e9-4292aeb12df0-kube-api-access-jjpzx\") pod \"odh-model-controller-7998c67dc4-2b76z\" (UID: \"c4af3523-66b5-4c1e-a6e9-4292aeb12df0\") " pod="opendatahub/odh-model-controller-7998c67dc4-2b76z"
Jul 10 02:16:29.265123 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.265095    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk"]
Jul 10 02:16:29.267918 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:16:29.267888    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cf43624_9057_462b_ba5e_e3a35ae43ec6.slice/crio-ea7200257ff507078b137f0e7039b52101acb3f67334b8b36de5826618f460da WatchSource:0}: Error finding container ea7200257ff507078b137f0e7039b52101acb3f67334b8b36de5826618f460da: Status 404 returned error can't find the container with id ea7200257ff507078b137f0e7039b52101acb3f67334b8b36de5826618f460da
Jul 10 02:16:29.725885 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.725845    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4af3523-66b5-4c1e-a6e9-4292aeb12df0-cert\") pod \"odh-model-controller-7998c67dc4-2b76z\" (UID: \"c4af3523-66b5-4c1e-a6e9-4292aeb12df0\") " pod="opendatahub/odh-model-controller-7998c67dc4-2b76z"
Jul 10 02:16:29.728209 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.728188    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4af3523-66b5-4c1e-a6e9-4292aeb12df0-cert\") pod \"odh-model-controller-7998c67dc4-2b76z\" (UID: \"c4af3523-66b5-4c1e-a6e9-4292aeb12df0\") " pod="opendatahub/odh-model-controller-7998c67dc4-2b76z"
Jul 10 02:16:29.856000 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.855962    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2b76z"
Jul 10 02:16:29.887305 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.887273    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hqtwr"]
Jul 10 02:16:29.890331 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.890311    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr"
Jul 10 02:16:29.892712 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.892690    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 10 02:16:29.892712 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.892706    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-kzdpw\""
Jul 10 02:16:29.905439 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:29.905412    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hqtwr"]
Jul 10 02:16:30.001718 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.001594    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2b76z"]
Jul 10 02:16:30.005127 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:16:30.005099    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4af3523_66b5_4c1e_a6e9_4292aeb12df0.slice/crio-5cb190ca00811307be82a1da8f2ffa1402594e2571fdc986bb1673ae072a2eb5 WatchSource:0}: Error finding container 5cb190ca00811307be82a1da8f2ffa1402594e2571fdc986bb1673ae072a2eb5: Status 404 returned error can't find the container with id 5cb190ca00811307be82a1da8f2ffa1402594e2571fdc986bb1673ae072a2eb5
Jul 10 02:16:30.017088 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.017064    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"]
Jul 10 02:16:30.020218 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.020199    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.021933 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.021914    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 10 02:16:30.022046 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.021958    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 10 02:16:30.022046 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.022025    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vj8ps\""
Jul 10 02:16:30.022192 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.022144    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 10 02:16:30.022259 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.022193    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 10 02:16:30.028564 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.028537    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4vrp\" (UniqueName: \"kubernetes.io/projected/217f9b2e-454b-4739-9d19-155d79101883-kube-api-access-j4vrp\") pod \"model-serving-api-54d89bd5fc-hqtwr\" (UID: \"217f9b2e-454b-4739-9d19-155d79101883\") " pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr"
Jul 10 02:16:30.028660 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.028592    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/217f9b2e-454b-4739-9d19-155d79101883-tls-certs\") pod \"model-serving-api-54d89bd5fc-hqtwr\" (UID: \"217f9b2e-454b-4739-9d19-155d79101883\") " pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr"
Jul 10 02:16:30.033312 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.033278    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"]
Jul 10 02:16:30.049036 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.049009    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2b76z" event={"ID":"c4af3523-66b5-4c1e-a6e9-4292aeb12df0","Type":"ContainerStarted","Data":"5cb190ca00811307be82a1da8f2ffa1402594e2571fdc986bb1673ae072a2eb5"}
Jul 10 02:16:30.049946 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.049929    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk" event={"ID":"2cf43624-9057-462b-ba5e-e3a35ae43ec6","Type":"ContainerStarted","Data":"ea7200257ff507078b137f0e7039b52101acb3f67334b8b36de5826618f460da"}
Jul 10 02:16:30.129528 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.129490    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/217f9b2e-454b-4739-9d19-155d79101883-tls-certs\") pod \"model-serving-api-54d89bd5fc-hqtwr\" (UID: \"217f9b2e-454b-4739-9d19-155d79101883\") " pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr"
Jul 10 02:16:30.129713 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.129559    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-token\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.129713 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.129598    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.129713 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.129628    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-local-certs\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.129713 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.129655    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmfwc\" (UniqueName: \"kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-kube-api-access-lmfwc\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.129947 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.129715    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.129947 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.129775    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-cacerts\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.129947 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.129840    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.129947 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.129886    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4vrp\" (UniqueName: \"kubernetes.io/projected/217f9b2e-454b-4739-9d19-155d79101883-kube-api-access-j4vrp\") pod \"model-serving-api-54d89bd5fc-hqtwr\" (UID: \"217f9b2e-454b-4739-9d19-155d79101883\") " pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr"
Jul 10 02:16:30.132445 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.132410    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/217f9b2e-454b-4739-9d19-155d79101883-tls-certs\") pod \"model-serving-api-54d89bd5fc-hqtwr\" (UID: \"217f9b2e-454b-4739-9d19-155d79101883\") " pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr"
Jul 10 02:16:30.138763 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.138737    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4vrp\" (UniqueName: \"kubernetes.io/projected/217f9b2e-454b-4739-9d19-155d79101883-kube-api-access-j4vrp\") pod \"model-serving-api-54d89bd5fc-hqtwr\" (UID: \"217f9b2e-454b-4739-9d19-155d79101883\") " pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr"
Jul 10 02:16:30.205152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.205113    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr"
Jul 10 02:16:30.230760 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.230732    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-local-certs\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.230921 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.230778    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmfwc\" (UniqueName: \"kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-kube-api-access-lmfwc\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.230921 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.230816    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.230921 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.230844    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-cacerts\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.231208 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.231153    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.231306 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.231267    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-token\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.231365 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.231306    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.231796 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.231769    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.233493 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.233468    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.233614 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.233592    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-local-certs\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.233676 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.233654    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.233722 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.233704    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-cacerts\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.239134 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.239111    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-token\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.239764 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.239734    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmfwc\" (UniqueName: \"kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-kube-api-access-lmfwc\") pod \"istiod-openshift-gateway-7574459656-wp7mc\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.336502 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.336468    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-hqtwr"]
Jul 10 02:16:30.339170 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.339147    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:30.339501 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:16:30.339466    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod217f9b2e_454b_4739_9d19_155d79101883.slice/crio-3da55edf3e11138b3f24343ba899061f418430851e9e4f412ca9aa3f8bcb1f4c WatchSource:0}: Error finding container 3da55edf3e11138b3f24343ba899061f418430851e9e4f412ca9aa3f8bcb1f4c: Status 404 returned error can't find the container with id 3da55edf3e11138b3f24343ba899061f418430851e9e4f412ca9aa3f8bcb1f4c
Jul 10 02:16:30.480735 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:30.480705    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"]
Jul 10 02:16:30.483770 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:16:30.483743    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d9b32cc_4a47_4c1d_b61d_309d541c4f4d.slice/crio-f0e30176f397518e0ac98f8db0556c3561a7258197ef64268af7c47f4053a31f WatchSource:0}: Error finding container f0e30176f397518e0ac98f8db0556c3561a7258197ef64268af7c47f4053a31f: Status 404 returned error can't find the container with id f0e30176f397518e0ac98f8db0556c3561a7258197ef64268af7c47f4053a31f
Jul 10 02:16:31.056496 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:31.056413    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr" event={"ID":"217f9b2e-454b-4739-9d19-155d79101883","Type":"ContainerStarted","Data":"3da55edf3e11138b3f24343ba899061f418430851e9e4f412ca9aa3f8bcb1f4c"}
Jul 10 02:16:31.060232 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:31.060171    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc" event={"ID":"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d","Type":"ContainerStarted","Data":"f0e30176f397518e0ac98f8db0556c3561a7258197ef64268af7c47f4053a31f"}
Jul 10 02:16:34.982910 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:34.982871    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:16:34.983257 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:34.982951    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:16:35.085639 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:35.085599    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc" event={"ID":"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d","Type":"ContainerStarted","Data":"1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e"}
Jul 10 02:16:35.085849 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:35.085692    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:35.087121 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:35.087089    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2b76z" event={"ID":"c4af3523-66b5-4c1e-a6e9-4292aeb12df0","Type":"ContainerStarted","Data":"352cc16ab6fcf9997d7245442e80f46ebdf10f9f0e36832e6a2d9a2547093018"}
Jul 10 02:16:35.087258 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:35.087214    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2b76z"
Jul 10 02:16:35.088550 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:35.088525    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk" event={"ID":"2cf43624-9057-462b-ba5e-e3a35ae43ec6","Type":"ContainerStarted","Data":"a452f426243e57be56d28ff753e19a069097a71dad2867da9b3cfb7a9cf735a0"}
Jul 10 02:16:35.089659 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:35.089639    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr" event={"ID":"217f9b2e-454b-4739-9d19-155d79101883","Type":"ContainerStarted","Data":"1240047ba1090db33632ae98162487a945a3d9083d492d734d3a346ce63610f9"}
Jul 10 02:16:35.089761 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:35.089749    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr"
Jul 10 02:16:35.126625 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:35.126550    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc" podStartSLOduration=1.62969867 podStartE2EDuration="6.126535757s" totalImagesPullingTime="4.496837087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:35.125580925 +0000 UTC m=+551.441890796" watchObservedRunningTime="2026-07-10 02:16:35.126535757 +0000 UTC m=+551.442845627"
Jul 10 02:16:35.156195 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:35.156139    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2b76z" podStartSLOduration=2.2314587120000002 podStartE2EDuration="7.15611907s" totalImagesPullingTime="4.924660358s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:35.154538778 +0000 UTC m=+551.470848671" watchObservedRunningTime="2026-07-10 02:16:35.15611907 +0000 UTC m=+551.472428987"
Jul 10 02:16:35.188294 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:35.188182    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr" podStartSLOduration=1.608307189 podStartE2EDuration="6.188163348s" totalImagesPullingTime="4.579856159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:35.187289674 +0000 UTC m=+551.503599560" watchObservedRunningTime="2026-07-10 02:16:35.188163348 +0000 UTC m=+551.504473220"
Jul 10 02:16:35.210194 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:35.210139    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6cf46c8b8c-7gbvk" podStartSLOduration=1.558805332 podStartE2EDuration="7.210120302s" totalImagesPullingTime="5.65131497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:35.209188365 +0000 UTC m=+551.525498257" watchObservedRunningTime="2026-07-10 02:16:35.210120302 +0000 UTC m=+551.526430176"
Jul 10 02:16:36.096113 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:36.096079    2557 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-wp7mc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 02:16:36.096484 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:36.096132    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc" podUID="1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:16:37.096654 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.096616    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"]
Jul 10 02:16:37.100025 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.100007    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.102090 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.102068    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-5r67p\""
Jul 10 02:16:37.112692 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.112669    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"]
Jul 10 02:16:37.196937 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.196882    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.196937 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.196924    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.197182 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.196945    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.197182 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.197016    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.197182 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.197056    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.197182 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.197078    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.197182 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.197112    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkfzp\" (UniqueName: \"kubernetes.io/projected/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-kube-api-access-gkfzp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.197432 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.197196    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.197432 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.197217    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.297573 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.297537    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.297573 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.297576    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.297830 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.297607    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkfzp\" (UniqueName: \"kubernetes.io/projected/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-kube-api-access-gkfzp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.297830 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.297649    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.297830 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.297679    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.297830 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.297701    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.297830 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.297745    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.297830 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.297775    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.297830 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.297827    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.298178 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.298009    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.298494 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.298467    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.298610 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.298489    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.298610 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.298575    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.298610 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.298584    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.300155 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.300129    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.300264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.300246    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.312556 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.312529    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.312779 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.312761    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkfzp\" (UniqueName: \"kubernetes.io/projected/6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec-kube-api-access-gkfzp\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46s2l9\" (UID: \"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.409672 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.409588    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:37.546720 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:37.546620    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"]
Jul 10 02:16:37.549140 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:16:37.549109    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e97fc62_5e41_4a29_bdfb_2af9d5e8e8ec.slice/crio-e862d77118a29d6108f6e9c2301617bddfaff48072f2fc736017730c35f8f4af WatchSource:0}: Error finding container e862d77118a29d6108f6e9c2301617bddfaff48072f2fc736017730c35f8f4af: Status 404 returned error can't find the container with id e862d77118a29d6108f6e9c2301617bddfaff48072f2fc736017730c35f8f4af
Jul 10 02:16:38.104148 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:38.104113    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9" event={"ID":"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec","Type":"ContainerStarted","Data":"e862d77118a29d6108f6e9c2301617bddfaff48072f2fc736017730c35f8f4af"}
Jul 10 02:16:39.012595 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.012564    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"]
Jul 10 02:16:39.015779 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.015759    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"
Jul 10 02:16:39.017787 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.017754    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-dsrpp\""
Jul 10 02:16:39.017787 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.017773    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 10 02:16:39.028445 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.028419    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"]
Jul 10 02:16:39.032590 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.032566    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"]
Jul 10 02:16:39.036179 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.036158    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:16:39.038146 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.038125    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-7xbbl\""
Jul 10 02:16:39.038146 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.038136    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 10 02:16:39.045658 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.045633    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"]
Jul 10 02:16:39.095965 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.095935    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:39.110698 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.110660    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9dfe9a84-a887-400c-9cd1-ebb4027492c4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nt97c\" (UID: \"9dfe9a84-a887-400c-9cd1-ebb4027492c4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:16:39.111147 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.110788    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw24b\" (UniqueName: \"kubernetes.io/projected/9dfe9a84-a887-400c-9cd1-ebb4027492c4-kube-api-access-kw24b\") pod \"llmisvc-controller-manager-777c6bd7d5-nt97c\" (UID: \"9dfe9a84-a887-400c-9cd1-ebb4027492c4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:16:39.111147 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.110889    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd63e164-0a54-474a-b508-978d78963df3-cert\") pod \"kserve-controller-manager-d6f4b65b8-4z6n8\" (UID: \"bd63e164-0a54-474a-b508-978d78963df3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"
Jul 10 02:16:39.111147 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.110923    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlhhh\" (UniqueName: \"kubernetes.io/projected/bd63e164-0a54-474a-b508-978d78963df3-kube-api-access-wlhhh\") pod \"kserve-controller-manager-d6f4b65b8-4z6n8\" (UID: \"bd63e164-0a54-474a-b508-978d78963df3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"
Jul 10 02:16:39.211413 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.211382    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9dfe9a84-a887-400c-9cd1-ebb4027492c4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nt97c\" (UID: \"9dfe9a84-a887-400c-9cd1-ebb4027492c4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:16:39.211607 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.211428    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw24b\" (UniqueName: \"kubernetes.io/projected/9dfe9a84-a887-400c-9cd1-ebb4027492c4-kube-api-access-kw24b\") pod \"llmisvc-controller-manager-777c6bd7d5-nt97c\" (UID: \"9dfe9a84-a887-400c-9cd1-ebb4027492c4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:16:39.211607 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:39.211542    2557 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 10 02:16:39.211607 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.211554    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd63e164-0a54-474a-b508-978d78963df3-cert\") pod \"kserve-controller-manager-d6f4b65b8-4z6n8\" (UID: \"bd63e164-0a54-474a-b508-978d78963df3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"
Jul 10 02:16:39.211607 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.211598    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlhhh\" (UniqueName: \"kubernetes.io/projected/bd63e164-0a54-474a-b508-978d78963df3-kube-api-access-wlhhh\") pod \"kserve-controller-manager-d6f4b65b8-4z6n8\" (UID: \"bd63e164-0a54-474a-b508-978d78963df3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"
Jul 10 02:16:39.211841 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:39.211614    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfe9a84-a887-400c-9cd1-ebb4027492c4-cert podName:9dfe9a84-a887-400c-9cd1-ebb4027492c4 nodeName:}" failed. No retries permitted until 2026-07-10 02:16:39.71159333 +0000 UTC m=+556.027903184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9dfe9a84-a887-400c-9cd1-ebb4027492c4-cert") pod "llmisvc-controller-manager-777c6bd7d5-nt97c" (UID: "9dfe9a84-a887-400c-9cd1-ebb4027492c4") : secret "llmisvc-webhook-server-cert" not found
Jul 10 02:16:39.214456 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.214428    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bd63e164-0a54-474a-b508-978d78963df3-cert\") pod \"kserve-controller-manager-d6f4b65b8-4z6n8\" (UID: \"bd63e164-0a54-474a-b508-978d78963df3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"
Jul 10 02:16:39.222922 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.222874    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlhhh\" (UniqueName: \"kubernetes.io/projected/bd63e164-0a54-474a-b508-978d78963df3-kube-api-access-wlhhh\") pod \"kserve-controller-manager-d6f4b65b8-4z6n8\" (UID: \"bd63e164-0a54-474a-b508-978d78963df3\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"
Jul 10 02:16:39.223244 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.223222    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw24b\" (UniqueName: \"kubernetes.io/projected/9dfe9a84-a887-400c-9cd1-ebb4027492c4-kube-api-access-kw24b\") pod \"llmisvc-controller-manager-777c6bd7d5-nt97c\" (UID: \"9dfe9a84-a887-400c-9cd1-ebb4027492c4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:16:39.328694 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.328656    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"
Jul 10 02:16:39.494582 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.494554    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"]
Jul 10 02:16:39.716499 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:39.716400    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9dfe9a84-a887-400c-9cd1-ebb4027492c4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nt97c\" (UID: \"9dfe9a84-a887-400c-9cd1-ebb4027492c4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:16:39.716639 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:39.716540    2557 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 10 02:16:39.716639 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:39.716612    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9dfe9a84-a887-400c-9cd1-ebb4027492c4-cert podName:9dfe9a84-a887-400c-9cd1-ebb4027492c4 nodeName:}" failed. No retries permitted until 2026-07-10 02:16:40.716594907 +0000 UTC m=+557.032904757 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9dfe9a84-a887-400c-9cd1-ebb4027492c4-cert") pod "llmisvc-controller-manager-777c6bd7d5-nt97c" (UID: "9dfe9a84-a887-400c-9cd1-ebb4027492c4") : secret "llmisvc-webhook-server-cert" not found
Jul 10 02:16:40.438443 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:16:40.438409    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd63e164_0a54_474a_b508_978d78963df3.slice/crio-63e581cf58dc5261adccb69f1a44f771d8edcb4355fd726bf668d2f249307d7f WatchSource:0}: Error finding container 63e581cf58dc5261adccb69f1a44f771d8edcb4355fd726bf668d2f249307d7f: Status 404 returned error can't find the container with id 63e581cf58dc5261adccb69f1a44f771d8edcb4355fd726bf668d2f249307d7f
Jul 10 02:16:40.497473 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:40.497439    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:16:40.497592 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:40.497510    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:16:40.497592 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:40.497538    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:16:40.723963 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:40.723872    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9dfe9a84-a887-400c-9cd1-ebb4027492c4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nt97c\" (UID: \"9dfe9a84-a887-400c-9cd1-ebb4027492c4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:16:40.726271 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:40.726245    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9dfe9a84-a887-400c-9cd1-ebb4027492c4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-nt97c\" (UID: \"9dfe9a84-a887-400c-9cd1-ebb4027492c4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:16:40.848524 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:40.848483    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:16:40.996425 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:40.996345    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"]
Jul 10 02:16:40.999699 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:16:40.999665    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9dfe9a84_a887_400c_9cd1_ebb4027492c4.slice/crio-e312d6225c9ca3922ca60504bd1060d8f8412a30998c6da95e2fee61ea638e2c WatchSource:0}: Error finding container e312d6225c9ca3922ca60504bd1060d8f8412a30998c6da95e2fee61ea638e2c: Status 404 returned error can't find the container with id e312d6225c9ca3922ca60504bd1060d8f8412a30998c6da95e2fee61ea638e2c
Jul 10 02:16:41.056459 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.056429    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"]
Jul 10 02:16:41.056607 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.056591    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc" podUID="1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" containerName="discovery" containerID="cri-o://1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e" gracePeriod=30
Jul 10 02:16:41.082439 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:41.082405    2557 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" image="quay.io/opendatahub/kserve-controller:odh-v3.5-EA2"
Jul 10 02:16:41.082579 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:41.082537    2557 kuberuntime_manager.go:1459] "Unhandled Error" err="container manager start failed in pod kserve-controller-manager-d6f4b65b8-4z6n8_opendatahub(bd63e164-0a54-474a-b508-978d78963df3): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image" logger="UnhandledError"
Jul 10 02:16:41.082625 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:41.082568    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8" podUID="bd63e164-0a54-474a-b508-978d78963df3"
Jul 10 02:16:41.118327 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.118297    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9" event={"ID":"6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec","Type":"ContainerStarted","Data":"44942417b18f46a13fa031128f0b8700d653a73c5e6d2b97ac1e63254d3228f9"}
Jul 10 02:16:41.119570 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.119544    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8" event={"ID":"bd63e164-0a54-474a-b508-978d78963df3","Type":"ContainerStarted","Data":"63e581cf58dc5261adccb69f1a44f771d8edcb4355fd726bf668d2f249307d7f"}
Jul 10 02:16:41.120086 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:41.120059    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/opendatahub/kserve-controller:odh-v3.5-EA2\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8" podUID="bd63e164-0a54-474a-b508-978d78963df3"
Jul 10 02:16:41.120667 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.120650    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c" event={"ID":"9dfe9a84-a887-400c-9cd1-ebb4027492c4","Type":"ContainerStarted","Data":"e312d6225c9ca3922ca60504bd1060d8f8412a30998c6da95e2fee61ea638e2c"}
Jul 10 02:16:41.152402 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.152355    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9" podStartSLOduration=1.206252232 podStartE2EDuration="4.152341689s" totalImagesPullingTime="2.946089457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:16:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:41.150936088 +0000 UTC m=+557.467245985" watchObservedRunningTime="2026-07-10 02:16:41.152341689 +0000 UTC m=+557.468651560"
Jul 10 02:16:41.292596 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.292567    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:41.328170 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.328138    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmfwc\" (UniqueName: \"kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-kube-api-access-lmfwc\") pod \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") "
Jul 10 02:16:41.328350 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.328186    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-kubeconfig\") pod \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") "
Jul 10 02:16:41.328350 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.328220    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-cacerts\") pod \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") "
Jul 10 02:16:41.328350 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.328251    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-dns-cert\") pod \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") "
Jul 10 02:16:41.328350 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.328312    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-local-certs\") pod \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") "
Jul 10 02:16:41.328350 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.328340    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-ca-configmap\") pod \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") "
Jul 10 02:16:41.328616 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.328368    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-token\") pod \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\" (UID: \"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d\") "
Jul 10 02:16:41.329041 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.328995    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" (UID: "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:16:41.330913 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.330882    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-token" (OuterVolumeSpecName: "istio-token") pod "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" (UID: "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:16:41.331017 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.330963    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" (UID: "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:41.331074 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.331011    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-cacerts" (OuterVolumeSpecName: "cacerts") pod "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" (UID: "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:41.331074 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.331011    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-local-certs" (OuterVolumeSpecName: "local-certs") pod "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" (UID: "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:16:41.331138 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.331073    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" (UID: "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:16:41.331171 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.331152    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-kube-api-access-lmfwc" (OuterVolumeSpecName: "kube-api-access-lmfwc") pod "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" (UID: "1d9b32cc-4a47-4c1d-b61d-309d541c4f4d"). InnerVolumeSpecName "kube-api-access-lmfwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:16:41.410481 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.410445    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:41.415049 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.415028    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:41.429346 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.429323    2557 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-cacerts\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:16:41.429346 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.429346    2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-dns-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:16:41.429467 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.429356    2557 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-local-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:16:41.429467 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.429365    2557 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-csr-ca-configmap\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:16:41.429467 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.429374    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:16:41.429467 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.429382    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmfwc\" (UniqueName: \"kubernetes.io/projected/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-kube-api-access-lmfwc\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:16:41.429467 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:41.429389    2557 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d-istio-kubeconfig\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:16:42.126040 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.126009    2557 generic.go:356] "Generic (PLEG): container finished" podID="1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" containerID="1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e" exitCode=0
Jul 10 02:16:42.126467 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.126073    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"
Jul 10 02:16:42.126467 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.126090    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc" event={"ID":"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d","Type":"ContainerDied","Data":"1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e"}
Jul 10 02:16:42.126467 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.126147    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc" event={"ID":"1d9b32cc-4a47-4c1d-b61d-309d541c4f4d","Type":"ContainerDied","Data":"f0e30176f397518e0ac98f8db0556c3561a7258197ef64268af7c47f4053a31f"}
Jul 10 02:16:42.126467 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.126170    2557 scope.go:117] "RemoveContainer" containerID="1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e"
Jul 10 02:16:42.126926 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.126852    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:42.127055 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:42.126923    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/opendatahub/kserve-controller:odh-v3.5-EA2\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: fetching blob: received unexpected HTTP status: 503 Service Unavailable; artifact err: provided artifact is a container image\"" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8" podUID="bd63e164-0a54-474a-b508-978d78963df3"
Jul 10 02:16:42.128061 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.128041    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46s2l9"
Jul 10 02:16:42.189317 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.189284    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"]
Jul 10 02:16:42.195640 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.195612    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wp7mc"]
Jul 10 02:16:42.209710 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.209682    2557 scope.go:117] "RemoveContainer" containerID="1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e"
Jul 10 02:16:42.210127 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:16:42.210105    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e\": container with ID starting with 1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e not found: ID does not exist" containerID="1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e"
Jul 10 02:16:42.210229 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.210142    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e"} err="failed to get container status \"1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e\": rpc error: code = NotFound desc = could not find container \"1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e\": container with ID starting with 1c535fb4cbfcc183b558fc7ecc6c64cbbe3b8d6cc2fe4553c83d406e4438c66e not found: ID does not exist"
Jul 10 02:16:42.241574 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:42.241545    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" path="/var/lib/kubelet/pods/1d9b32cc-4a47-4c1d-b61d-309d541c4f4d/volumes"
Jul 10 02:16:45.140282 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:45.140242    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c" event={"ID":"9dfe9a84-a887-400c-9cd1-ebb4027492c4","Type":"ContainerStarted","Data":"5539ecf3f0d2c99d286d0b6d54b3246041a7c9cbaa43dee9ee46a6702890469f"}
Jul 10 02:16:45.140644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:45.140369    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:16:45.159525 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:45.159480    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c" podStartSLOduration=2.988712407 podStartE2EDuration="6.159465869s" totalImagesPullingTime="3.170753462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:16:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:16:45.157971482 +0000 UTC m=+561.474281357" watchObservedRunningTime="2026-07-10 02:16:45.159465869 +0000 UTC m=+561.475775761"
Jul 10 02:16:46.096956 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:46.096927    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2b76z"
Jul 10 02:16:46.099144 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:46.099124    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-hqtwr"
Jul 10 02:16:59.198190 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:59.198102    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8" event={"ID":"bd63e164-0a54-474a-b508-978d78963df3","Type":"ContainerStarted","Data":"5406060967c698e803d8e35a2c877158884e7343a4fc840e42b84dfe5df2024d"}
Jul 10 02:16:59.198553 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:59.198322    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"
Jul 10 02:16:59.214538 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:16:59.214488    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8" podStartSLOduration=2.753964044 podStartE2EDuration="21.214473227s" totalImagesPullingTime="18.460509183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:16:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-10 02:16:59.213103278 +0000 UTC m=+575.529413161" watchObservedRunningTime="2026-07-10 02:16:59.214473227 +0000 UTC m=+575.530783121"
Jul 10 02:17:16.146520 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:16.146491    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-nt97c"
Jul 10 02:17:24.126647 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:24.126612    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:17:24.127871 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:24.127845    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:17:30.207508 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:30.207476    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-4z6n8"
Jul 10 02:17:50.035079 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.034994    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b5xld"]
Jul 10 02:17:50.035632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.035272    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" containerName="discovery"
Jul 10 02:17:50.035632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.035283    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" containerName="discovery"
Jul 10 02:17:50.035632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.035333    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d9b32cc-4a47-4c1d-b61d-309d541c4f4d" containerName="discovery"
Jul 10 02:17:50.038284 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.038260    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b5xld"
Jul 10 02:17:50.040460 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.040437    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 10 02:17:50.040609 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.040456    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-9r6vt\""
Jul 10 02:17:50.040609 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.040491    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 10 02:17:50.046112 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.045904    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b5xld"]
Jul 10 02:17:50.069002 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.068963    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvsz8\" (UniqueName: \"kubernetes.io/projected/67a47d1e-41ee-42b3-98b8-7d95cca19905-kube-api-access-tvsz8\") pod \"kuadrant-operator-catalog-b5xld\" (UID: \"67a47d1e-41ee-42b3-98b8-7d95cca19905\") " pod="kuadrant-system/kuadrant-operator-catalog-b5xld"
Jul 10 02:17:50.169496 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.169458    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvsz8\" (UniqueName: \"kubernetes.io/projected/67a47d1e-41ee-42b3-98b8-7d95cca19905-kube-api-access-tvsz8\") pod \"kuadrant-operator-catalog-b5xld\" (UID: \"67a47d1e-41ee-42b3-98b8-7d95cca19905\") " pod="kuadrant-system/kuadrant-operator-catalog-b5xld"
Jul 10 02:17:50.178508 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.178473    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvsz8\" (UniqueName: \"kubernetes.io/projected/67a47d1e-41ee-42b3-98b8-7d95cca19905-kube-api-access-tvsz8\") pod \"kuadrant-operator-catalog-b5xld\" (UID: \"67a47d1e-41ee-42b3-98b8-7d95cca19905\") " pod="kuadrant-system/kuadrant-operator-catalog-b5xld"
Jul 10 02:17:50.349048 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.348960    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b5xld"
Jul 10 02:17:50.392218 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.392177    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b5xld"]
Jul 10 02:17:50.477224 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.477193    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b5xld"]
Jul 10 02:17:50.480331 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:17:50.480303    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a47d1e_41ee_42b3_98b8_7d95cca19905.slice/crio-d38dc71f37017bcf583fba47062e544659e75c626aa0596f10de725072c6b346 WatchSource:0}: Error finding container d38dc71f37017bcf583fba47062e544659e75c626aa0596f10de725072c6b346: Status 404 returned error can't find the container with id d38dc71f37017bcf583fba47062e544659e75c626aa0596f10de725072c6b346
Jul 10 02:17:50.481852 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.481831    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:17:50.601890 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.601790    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hjwmj"]
Jul 10 02:17:50.606108 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.606085    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hjwmj"
Jul 10 02:17:50.613030 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.613001    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hjwmj"]
Jul 10 02:17:50.673170 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.673137    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqcv5\" (UniqueName: \"kubernetes.io/projected/7d4ae36e-5089-43b4-acd9-d9ce955e26bf-kube-api-access-tqcv5\") pod \"kuadrant-operator-catalog-hjwmj\" (UID: \"7d4ae36e-5089-43b4-acd9-d9ce955e26bf\") " pod="kuadrant-system/kuadrant-operator-catalog-hjwmj"
Jul 10 02:17:50.774078 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.774043    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqcv5\" (UniqueName: \"kubernetes.io/projected/7d4ae36e-5089-43b4-acd9-d9ce955e26bf-kube-api-access-tqcv5\") pod \"kuadrant-operator-catalog-hjwmj\" (UID: \"7d4ae36e-5089-43b4-acd9-d9ce955e26bf\") " pod="kuadrant-system/kuadrant-operator-catalog-hjwmj"
Jul 10 02:17:50.783692 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.783669    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqcv5\" (UniqueName: \"kubernetes.io/projected/7d4ae36e-5089-43b4-acd9-d9ce955e26bf-kube-api-access-tqcv5\") pod \"kuadrant-operator-catalog-hjwmj\" (UID: \"7d4ae36e-5089-43b4-acd9-d9ce955e26bf\") " pod="kuadrant-system/kuadrant-operator-catalog-hjwmj"
Jul 10 02:17:50.916608 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:50.916524    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hjwmj"
Jul 10 02:17:51.045292 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:51.045263    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hjwmj"]
Jul 10 02:17:51.072904 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:17:51.072876    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d4ae36e_5089_43b4_acd9_d9ce955e26bf.slice/crio-09a7e94b3cbe7920d410967edfed2668e5a10bc4d9b060e8958dec08ffb8dfcc WatchSource:0}: Error finding container 09a7e94b3cbe7920d410967edfed2668e5a10bc4d9b060e8958dec08ffb8dfcc: Status 404 returned error can't find the container with id 09a7e94b3cbe7920d410967edfed2668e5a10bc4d9b060e8958dec08ffb8dfcc
Jul 10 02:17:51.403123 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:51.403089    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hjwmj" event={"ID":"7d4ae36e-5089-43b4-acd9-d9ce955e26bf","Type":"ContainerStarted","Data":"09a7e94b3cbe7920d410967edfed2668e5a10bc4d9b060e8958dec08ffb8dfcc"}
Jul 10 02:17:51.404296 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:51.404270    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b5xld" event={"ID":"67a47d1e-41ee-42b3-98b8-7d95cca19905","Type":"ContainerStarted","Data":"d38dc71f37017bcf583fba47062e544659e75c626aa0596f10de725072c6b346"}
Jul 10 02:17:53.415136 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:53.415097    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hjwmj" event={"ID":"7d4ae36e-5089-43b4-acd9-d9ce955e26bf","Type":"ContainerStarted","Data":"5053d41246339e165a5684424ab9b716a2aef0520783e8acaac5cfab0b170b8e"}
Jul 10 02:17:53.416484 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:53.416459    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b5xld" event={"ID":"67a47d1e-41ee-42b3-98b8-7d95cca19905","Type":"ContainerStarted","Data":"bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0"}
Jul 10 02:17:53.416587 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:53.416515    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-b5xld" podUID="67a47d1e-41ee-42b3-98b8-7d95cca19905" containerName="registry-server" containerID="cri-o://bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0" gracePeriod=2
Jul 10 02:17:53.432599 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:53.432556    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hjwmj" podStartSLOduration=1.960645578 podStartE2EDuration="3.432543584s" totalImagesPullingTime="1.471898006s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:17:53.43216606 +0000 UTC m=+629.748475956" watchObservedRunningTime="2026-07-10 02:17:53.432543584 +0000 UTC m=+629.748853455"
Jul 10 02:17:53.449785 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:53.449746    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-b5xld" podStartSLOduration=1.3889176189999999 podStartE2EDuration="3.449733622s" totalImagesPullingTime="2.060816003s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:17:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:17:53.448842976 +0000 UTC m=+629.765152869" watchObservedRunningTime="2026-07-10 02:17:53.449733622 +0000 UTC m=+629.766043823"
Jul 10 02:17:53.656557 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:53.656534    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b5xld"
Jul 10 02:17:53.699060 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:53.698984    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvsz8\" (UniqueName: \"kubernetes.io/projected/67a47d1e-41ee-42b3-98b8-7d95cca19905-kube-api-access-tvsz8\") pod \"67a47d1e-41ee-42b3-98b8-7d95cca19905\" (UID: \"67a47d1e-41ee-42b3-98b8-7d95cca19905\") "
Jul 10 02:17:53.701094 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:53.701073    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67a47d1e-41ee-42b3-98b8-7d95cca19905-kube-api-access-tvsz8" (OuterVolumeSpecName: "kube-api-access-tvsz8") pod "67a47d1e-41ee-42b3-98b8-7d95cca19905" (UID: "67a47d1e-41ee-42b3-98b8-7d95cca19905"). InnerVolumeSpecName "kube-api-access-tvsz8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:17:53.799762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:53.799719    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvsz8\" (UniqueName: \"kubernetes.io/projected/67a47d1e-41ee-42b3-98b8-7d95cca19905-kube-api-access-tvsz8\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:17:54.421291 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:54.421261    2557 generic.go:356] "Generic (PLEG): container finished" podID="67a47d1e-41ee-42b3-98b8-7d95cca19905" containerID="bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0" exitCode=0
Jul 10 02:17:54.421705 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:54.421305    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-b5xld"
Jul 10 02:17:54.421705 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:54.421347    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b5xld" event={"ID":"67a47d1e-41ee-42b3-98b8-7d95cca19905","Type":"ContainerDied","Data":"bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0"}
Jul 10 02:17:54.421705 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:54.421394    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-b5xld" event={"ID":"67a47d1e-41ee-42b3-98b8-7d95cca19905","Type":"ContainerDied","Data":"d38dc71f37017bcf583fba47062e544659e75c626aa0596f10de725072c6b346"}
Jul 10 02:17:54.421705 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:54.421420    2557 scope.go:117] "RemoveContainer" containerID="bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0"
Jul 10 02:17:54.429654 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:54.429638    2557 scope.go:117] "RemoveContainer" containerID="bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0"
Jul 10 02:17:54.429913 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:17:54.429896    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0\": container with ID starting with bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0 not found: ID does not exist" containerID="bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0"
Jul 10 02:17:54.429983 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:54.429923    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0"} err="failed to get container status \"bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0\": rpc error: code = NotFound desc = could not find container \"bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0\": container with ID starting with bfb6bbb83220a02bdc8f16b2c1e21577e3d69ab40264038eac9dff574dc596f0 not found: ID does not exist"
Jul 10 02:17:54.438670 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:54.438647    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b5xld"]
Jul 10 02:17:54.443612 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:54.443593    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-b5xld"]
Jul 10 02:17:56.241694 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:17:56.241662    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67a47d1e-41ee-42b3-98b8-7d95cca19905" path="/var/lib/kubelet/pods/67a47d1e-41ee-42b3-98b8-7d95cca19905/volumes"
Jul 10 02:18:00.917442 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:00.917408    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-hjwmj"
Jul 10 02:18:00.917877 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:00.917458    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-hjwmj"
Jul 10 02:18:00.939061 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:00.939025    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-hjwmj"
Jul 10 02:18:01.473353 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:01.473328    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-hjwmj"
Jul 10 02:18:22.495331 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.495295    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"]
Jul 10 02:18:22.495829 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.495589    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67a47d1e-41ee-42b3-98b8-7d95cca19905" containerName="registry-server"
Jul 10 02:18:22.495829 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.495601    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="67a47d1e-41ee-42b3-98b8-7d95cca19905" containerName="registry-server"
Jul 10 02:18:22.495829 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.495671    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="67a47d1e-41ee-42b3-98b8-7d95cca19905" containerName="registry-server"
Jul 10 02:18:22.502072 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.502052    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:22.504225 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.504199    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-gg6f2\""
Jul 10 02:18:22.510407 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.510386    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"]
Jul 10 02:18:22.629062 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.629028    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlqzr\" (UniqueName: \"kubernetes.io/projected/bcd6120d-cce0-4824-8c6d-ca0d5a7f4578-kube-api-access-vlqzr\") pod \"limitador-operator-controller-manager-859d7b78bd-55dhc\" (UID: \"bcd6120d-cce0-4824-8c6d-ca0d5a7f4578\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:22.729640 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.729607    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlqzr\" (UniqueName: \"kubernetes.io/projected/bcd6120d-cce0-4824-8c6d-ca0d5a7f4578-kube-api-access-vlqzr\") pod \"limitador-operator-controller-manager-859d7b78bd-55dhc\" (UID: \"bcd6120d-cce0-4824-8c6d-ca0d5a7f4578\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:22.742210 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.742178    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlqzr\" (UniqueName: \"kubernetes.io/projected/bcd6120d-cce0-4824-8c6d-ca0d5a7f4578-kube-api-access-vlqzr\") pod \"limitador-operator-controller-manager-859d7b78bd-55dhc\" (UID: \"bcd6120d-cce0-4824-8c6d-ca0d5a7f4578\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:22.813636 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.813606    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:22.941421 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:22.941396    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"]
Jul 10 02:18:22.943881 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:18:22.943855    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd6120d_cce0_4824_8c6d_ca0d5a7f4578.slice/crio-2f63977693185f256f674c8e8066acf6be8abaac7b61dfabcc9c7ad6dd5b8043 WatchSource:0}: Error finding container 2f63977693185f256f674c8e8066acf6be8abaac7b61dfabcc9c7ad6dd5b8043: Status 404 returned error can't find the container with id 2f63977693185f256f674c8e8066acf6be8abaac7b61dfabcc9c7ad6dd5b8043
Jul 10 02:18:23.543117 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:23.543078    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc" event={"ID":"bcd6120d-cce0-4824-8c6d-ca0d5a7f4578","Type":"ContainerStarted","Data":"2f63977693185f256f674c8e8066acf6be8abaac7b61dfabcc9c7ad6dd5b8043"}
Jul 10 02:18:25.553830 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:25.553783    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc" event={"ID":"bcd6120d-cce0-4824-8c6d-ca0d5a7f4578","Type":"ContainerStarted","Data":"20524893b5c704de08dfa166f4d6ac49d8bf5594ed64419742e5ad1cc17896ce"}
Jul 10 02:18:25.554267 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:25.553903    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:25.571355 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:25.571311    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc" podStartSLOduration=1.770261232 podStartE2EDuration="3.571298661s" totalImagesPullingTime="1.801037429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:18:25.570239452 +0000 UTC m=+661.886549334" watchObservedRunningTime="2026-07-10 02:18:25.571298661 +0000 UTC m=+661.887608533"
Jul 10 02:18:27.674341 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:27.674308    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"]
Jul 10 02:18:27.677628 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:27.677611    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:27.680177 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:27.680153    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-w5jbk\""
Jul 10 02:18:27.689317 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:27.689296    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"]
Jul 10 02:18:27.776395 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:27.776360    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szm2v\" (UID: \"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:27.776395 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:27.776400    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms6c2\" (UniqueName: \"kubernetes.io/projected/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-kube-api-access-ms6c2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szm2v\" (UID: \"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:27.877627 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:27.877588    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szm2v\" (UID: \"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:27.877798 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:27.877639    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms6c2\" (UniqueName: \"kubernetes.io/projected/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-kube-api-access-ms6c2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szm2v\" (UID: \"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:27.877985 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:27.877964    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szm2v\" (UID: \"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:27.887663 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:27.887634    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms6c2\" (UniqueName: \"kubernetes.io/projected/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-kube-api-access-ms6c2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-szm2v\" (UID: \"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:27.987589 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:27.987504    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:28.116931 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:28.116909    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"]
Jul 10 02:18:28.119208 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:18:28.119182    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod432f2b99_bc4e_4c76_bcc8_0f7a4f2a46fc.slice/crio-867a1d56c4efcb8e285b322e79ae494cdb983c705e76811ab6cd886a54098896 WatchSource:0}: Error finding container 867a1d56c4efcb8e285b322e79ae494cdb983c705e76811ab6cd886a54098896: Status 404 returned error can't find the container with id 867a1d56c4efcb8e285b322e79ae494cdb983c705e76811ab6cd886a54098896
Jul 10 02:18:28.567529 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:28.567495    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v" event={"ID":"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc","Type":"ContainerStarted","Data":"867a1d56c4efcb8e285b322e79ae494cdb983c705e76811ab6cd886a54098896"}
Jul 10 02:18:30.280246 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:30.280204    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-m94kz"]
Jul 10 02:18:30.283932 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:30.283908    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m94kz"
Jul 10 02:18:30.285797 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:30.285765    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-npksg\""
Jul 10 02:18:30.292485 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:30.292457    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-m94kz"]
Jul 10 02:18:30.399563 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:30.399523    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkdrj\" (UniqueName: \"kubernetes.io/projected/abaf9abf-1334-4a5f-a143-961d7da59892-kube-api-access-nkdrj\") pod \"authorino-operator-75cc8d5cb7-m94kz\" (UID: \"abaf9abf-1334-4a5f-a143-961d7da59892\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-m94kz"
Jul 10 02:18:30.501064 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:30.501016    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkdrj\" (UniqueName: \"kubernetes.io/projected/abaf9abf-1334-4a5f-a143-961d7da59892-kube-api-access-nkdrj\") pod \"authorino-operator-75cc8d5cb7-m94kz\" (UID: \"abaf9abf-1334-4a5f-a143-961d7da59892\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-m94kz"
Jul 10 02:18:30.510726 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:30.510689    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkdrj\" (UniqueName: \"kubernetes.io/projected/abaf9abf-1334-4a5f-a143-961d7da59892-kube-api-access-nkdrj\") pod \"authorino-operator-75cc8d5cb7-m94kz\" (UID: \"abaf9abf-1334-4a5f-a143-961d7da59892\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-m94kz"
Jul 10 02:18:30.597596 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:30.597520    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m94kz"
Jul 10 02:18:30.724070 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:30.724036    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-m94kz"]
Jul 10 02:18:30.726906 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:18:30.726880    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabaf9abf_1334_4a5f_a143_961d7da59892.slice/crio-3e459f23dc175a255f18fb2c93d9d242253b23982b6e17306182bfb67975ce94 WatchSource:0}: Error finding container 3e459f23dc175a255f18fb2c93d9d242253b23982b6e17306182bfb67975ce94: Status 404 returned error can't find the container with id 3e459f23dc175a255f18fb2c93d9d242253b23982b6e17306182bfb67975ce94
Jul 10 02:18:31.581956 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:31.581920    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m94kz" event={"ID":"abaf9abf-1334-4a5f-a143-961d7da59892","Type":"ContainerStarted","Data":"3e459f23dc175a255f18fb2c93d9d242253b23982b6e17306182bfb67975ce94"}
Jul 10 02:18:34.598052 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:34.598018    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v" event={"ID":"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc","Type":"ContainerStarted","Data":"2f25306f8a5547d7be1bf0fa1f6a21915527e361a5da088da29523c5f3af6225"}
Jul 10 02:18:34.598471 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:34.598116    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:34.599660 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:34.599638    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m94kz" event={"ID":"abaf9abf-1334-4a5f-a143-961d7da59892","Type":"ContainerStarted","Data":"b514a0fbce562ff056603994144a3abf3380f0f4cbb8c604488216b4886713dd"}
Jul 10 02:18:34.599787 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:34.599759    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m94kz"
Jul 10 02:18:34.617688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:34.617634    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v" podStartSLOduration=1.329971777 podStartE2EDuration="7.617614306s" totalImagesPullingTime="6.287642529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:18:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:18:34.61615412 +0000 UTC m=+670.932464004" watchObservedRunningTime="2026-07-10 02:18:34.617614306 +0000 UTC m=+670.933924179"
Jul 10 02:18:34.633683 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:34.633620    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m94kz" podStartSLOduration=0.963419891 podStartE2EDuration="4.633602674s" totalImagesPullingTime="3.670182783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:18:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:18:34.63219454 +0000 UTC m=+670.948504412" watchObservedRunningTime="2026-07-10 02:18:34.633602674 +0000 UTC m=+670.949912547"
Jul 10 02:18:36.560955 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:36.560921    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:45.605573 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:45.605534    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:45.605966 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:45.605621    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-m94kz"
Jul 10 02:18:47.478555 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.478515    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"]
Jul 10 02:18:47.478969 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.478744    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v" podUID="432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" containerName="manager" containerID="cri-o://2f25306f8a5547d7be1bf0fa1f6a21915527e361a5da088da29523c5f3af6225" gracePeriod=2
Jul 10 02:18:47.487888 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.487863    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"]
Jul 10 02:18:47.500488 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.500466    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"]
Jul 10 02:18:47.500667 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.500646    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc" podUID="bcd6120d-cce0-4824-8c6d-ca0d5a7f4578" containerName="manager" containerID="cri-o://20524893b5c704de08dfa166f4d6ac49d8bf5594ed64419742e5ad1cc17896ce" gracePeriod=2
Jul 10 02:18:47.502468 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:18:47.502441    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-szm2v\" is forbidden: User \"system:node:ip-10-0-131-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-213.ec2.internal' and this object" podUID="432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:47.527080 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.527053    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"]
Jul 10 02:18:47.527476 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.527458    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" containerName="manager"
Jul 10 02:18:47.527524 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.527481    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" containerName="manager"
Jul 10 02:18:47.527605 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.527592    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" containerName="manager"
Jul 10 02:18:47.530515 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.530498    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:18:47.538209 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.538185    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"]
Jul 10 02:18:47.540249 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:18:47.540217    2557 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bcd6120d-cce0-4824-8c6d-ca0d5a7f4578\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-55dhc\": pods \"limitador-operator-controller-manager-859d7b78bd-55dhc\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:47.551579 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.551443    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"]
Jul 10 02:18:47.570637 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:18:47.570598    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-szm2v\" is forbidden: User \"system:node:ip-10-0-131-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-213.ec2.internal' and this object" podUID="432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:47.572372 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:18:47.572346    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-55dhc\" is forbidden: User \"system:node:ip-10-0-131-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-213.ec2.internal' and this object" podUID="bcd6120d-cce0-4824-8c6d-ca0d5a7f4578" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:47.634550 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.634518    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dkgv\" (UniqueName: \"kubernetes.io/projected/b37484bc-6764-46ac-9e9f-9a4a6368e355-kube-api-access-8dkgv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sltx2\" (UID: \"b37484bc-6764-46ac-9e9f-9a4a6368e355\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:18:47.634692 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.634566    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b37484bc-6764-46ac-9e9f-9a4a6368e355-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sltx2\" (UID: \"b37484bc-6764-46ac-9e9f-9a4a6368e355\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:18:47.657640 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.657610    2557 generic.go:356] "Generic (PLEG): container finished" podID="bcd6120d-cce0-4824-8c6d-ca0d5a7f4578" containerID="20524893b5c704de08dfa166f4d6ac49d8bf5594ed64419742e5ad1cc17896ce" exitCode=0
Jul 10 02:18:47.659173 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.659150    2557 generic.go:356] "Generic (PLEG): container finished" podID="432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" containerID="2f25306f8a5547d7be1bf0fa1f6a21915527e361a5da088da29523c5f3af6225" exitCode=0
Jul 10 02:18:47.735993 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.735967    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b37484bc-6764-46ac-9e9f-9a4a6368e355-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sltx2\" (UID: \"b37484bc-6764-46ac-9e9f-9a4a6368e355\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:18:47.736082 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.736042    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dkgv\" (UniqueName: \"kubernetes.io/projected/b37484bc-6764-46ac-9e9f-9a4a6368e355-kube-api-access-8dkgv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sltx2\" (UID: \"b37484bc-6764-46ac-9e9f-9a4a6368e355\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:18:47.736429 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.736406    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b37484bc-6764-46ac-9e9f-9a4a6368e355-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sltx2\" (UID: \"b37484bc-6764-46ac-9e9f-9a4a6368e355\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:18:47.749433 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.749409    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:47.749551 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.749490    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dkgv\" (UniqueName: \"kubernetes.io/projected/b37484bc-6764-46ac-9e9f-9a4a6368e355-kube-api-access-8dkgv\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sltx2\" (UID: \"b37484bc-6764-46ac-9e9f-9a4a6368e355\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:18:47.751508 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:18:47.751486    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-szm2v\" is forbidden: User \"system:node:ip-10-0-131-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-213.ec2.internal' and this object" podUID="432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:47.751884 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.751869    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:47.753215 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:18:47.753198    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-55dhc\" is forbidden: User \"system:node:ip-10-0-131-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-213.ec2.internal' and this object" podUID="bcd6120d-cce0-4824-8c6d-ca0d5a7f4578" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:47.754967 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:18:47.754942    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-szm2v\" is forbidden: User \"system:node:ip-10-0-131-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-213.ec2.internal' and this object" podUID="432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:47.756645 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:18:47.756621    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-55dhc\" is forbidden: User \"system:node:ip-10-0-131-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-213.ec2.internal' and this object" podUID="bcd6120d-cce0-4824-8c6d-ca0d5a7f4578" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:47.836713 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.836680    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlqzr\" (UniqueName: \"kubernetes.io/projected/bcd6120d-cce0-4824-8c6d-ca0d5a7f4578-kube-api-access-vlqzr\") pod \"bcd6120d-cce0-4824-8c6d-ca0d5a7f4578\" (UID: \"bcd6120d-cce0-4824-8c6d-ca0d5a7f4578\") "
Jul 10 02:18:47.836713 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.836719    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-extensions-socket-volume\") pod \"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc\" (UID: \"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc\") "
Jul 10 02:18:47.836933 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.836760    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ms6c2\" (UniqueName: \"kubernetes.io/projected/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-kube-api-access-ms6c2\") pod \"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc\" (UID: \"432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc\") "
Jul 10 02:18:47.837232 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.837209    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" (UID: "432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:18:47.838895 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.838866    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-kube-api-access-ms6c2" (OuterVolumeSpecName: "kube-api-access-ms6c2") pod "432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" (UID: "432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc"). InnerVolumeSpecName "kube-api-access-ms6c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:18:47.838956 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.838898    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcd6120d-cce0-4824-8c6d-ca0d5a7f4578-kube-api-access-vlqzr" (OuterVolumeSpecName: "kube-api-access-vlqzr") pod "bcd6120d-cce0-4824-8c6d-ca0d5a7f4578" (UID: "bcd6120d-cce0-4824-8c6d-ca0d5a7f4578"). InnerVolumeSpecName "kube-api-access-vlqzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:18:47.922141 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.922092    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:18:47.937677 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.937654    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms6c2\" (UniqueName: \"kubernetes.io/projected/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-kube-api-access-ms6c2\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:18:47.937755 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.937680    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlqzr\" (UniqueName: \"kubernetes.io/projected/bcd6120d-cce0-4824-8c6d-ca0d5a7f4578-kube-api-access-vlqzr\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:18:47.937755 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:47.937690    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc-extensions-socket-volume\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:18:48.078994 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.078963    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"]
Jul 10 02:18:48.081668 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:18:48.081633    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb37484bc_6764_46ac_9e9f_9a4a6368e355.slice/crio-cd688d5689fbfbf15e42dcf40af8110a41a87c6226bb10cbf638b15dc98f1855 WatchSource:0}: Error finding container cd688d5689fbfbf15e42dcf40af8110a41a87c6226bb10cbf638b15dc98f1855: Status 404 returned error can't find the container with id cd688d5689fbfbf15e42dcf40af8110a41a87c6226bb10cbf638b15dc98f1855
Jul 10 02:18:48.241520 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.241485    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc" path="/var/lib/kubelet/pods/432f2b99-bc4e-4c76-bcc8-0f7a4f2a46fc/volumes"
Jul 10 02:18:48.241812 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.241788    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcd6120d-cce0-4824-8c6d-ca0d5a7f4578" path="/var/lib/kubelet/pods/bcd6120d-cce0-4824-8c6d-ca0d5a7f4578/volumes"
Jul 10 02:18:48.405071 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.405040    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"]
Jul 10 02:18:48.405348 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.405336    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcd6120d-cce0-4824-8c6d-ca0d5a7f4578" containerName="manager"
Jul 10 02:18:48.405391 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.405349    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcd6120d-cce0-4824-8c6d-ca0d5a7f4578" containerName="manager"
Jul 10 02:18:48.405421 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.405406    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcd6120d-cce0-4824-8c6d-ca0d5a7f4578" containerName="manager"
Jul 10 02:18:48.408597 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.408577    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:18:48.421378 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.421353    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"]
Jul 10 02:18:48.442465 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.442441    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw87x\" (UniqueName: \"kubernetes.io/projected/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-kube-api-access-gw87x\") pod \"kuadrant-operator-controller-manager-86878c769f-xl44w\" (UID: \"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:18:48.442570 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.442554    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xl44w\" (UID: \"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:18:48.543412 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.543375    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xl44w\" (UID: \"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:18:48.543796 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.543430    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw87x\" (UniqueName: \"kubernetes.io/projected/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-kube-api-access-gw87x\") pod \"kuadrant-operator-controller-manager-86878c769f-xl44w\" (UID: \"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:18:48.543796 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.543780    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-xl44w\" (UID: \"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:18:48.555323 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.555291    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw87x\" (UniqueName: \"kubernetes.io/projected/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-kube-api-access-gw87x\") pod \"kuadrant-operator-controller-manager-86878c769f-xl44w\" (UID: \"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:18:48.666247 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.666213    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2" event={"ID":"b37484bc-6764-46ac-9e9f-9a4a6368e355","Type":"ContainerStarted","Data":"b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e"}
Jul 10 02:18:48.666422 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.666264    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2" event={"ID":"b37484bc-6764-46ac-9e9f-9a4a6368e355","Type":"ContainerStarted","Data":"cd688d5689fbfbf15e42dcf40af8110a41a87c6226bb10cbf638b15dc98f1855"}
Jul 10 02:18:48.666422 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.666311    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:18:48.667602 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.667582    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-55dhc"
Jul 10 02:18:48.667704 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.667609    2557 scope.go:117] "RemoveContainer" containerID="20524893b5c704de08dfa166f4d6ac49d8bf5594ed64419742e5ad1cc17896ce"
Jul 10 02:18:48.669163 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.669110    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-szm2v"
Jul 10 02:18:48.676347 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.676326    2557 scope.go:117] "RemoveContainer" containerID="2f25306f8a5547d7be1bf0fa1f6a21915527e361a5da088da29523c5f3af6225"
Jul 10 02:18:48.694432 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.694387    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2" podStartSLOduration=1.694369333 podStartE2EDuration="1.694369333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:18:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:18:48.69305876 +0000 UTC m=+685.009368657" watchObservedRunningTime="2026-07-10 02:18:48.694369333 +0000 UTC m=+685.010679208"
Jul 10 02:18:48.717965 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.717936    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:18:48.855567 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:48.855498    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"]
Jul 10 02:18:48.858274 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:18:48.858248    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d4d7cdd_d147_45b4_8141_e9d79b5d98a6.slice/crio-eadd63e5c003a61a5470408f6eae54d5fe3e8ecf909621da093fac2d5b18605a WatchSource:0}: Error finding container eadd63e5c003a61a5470408f6eae54d5fe3e8ecf909621da093fac2d5b18605a: Status 404 returned error can't find the container with id eadd63e5c003a61a5470408f6eae54d5fe3e8ecf909621da093fac2d5b18605a
Jul 10 02:18:49.678017 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:49.677979    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w" event={"ID":"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6","Type":"ContainerStarted","Data":"8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc"}
Jul 10 02:18:49.678422 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:49.678030    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w" event={"ID":"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6","Type":"ContainerStarted","Data":"eadd63e5c003a61a5470408f6eae54d5fe3e8ecf909621da093fac2d5b18605a"}
Jul 10 02:18:49.678422 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:49.678056    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:18:49.700097 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:49.700051    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w" podStartSLOduration=1.700033016 podStartE2EDuration="1.700033016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:18:49.698174512 +0000 UTC m=+686.014484406" watchObservedRunningTime="2026-07-10 02:18:49.700033016 +0000 UTC m=+686.016342889"
Jul 10 02:18:59.679897 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:18:59.679867    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:19:00.685263 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:00.685233    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:19:00.751184 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:00.751151    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"]
Jul 10 02:19:00.751420 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:00.751376    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2" podUID="b37484bc-6764-46ac-9e9f-9a4a6368e355" containerName="manager" containerID="cri-o://b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e" gracePeriod=10
Jul 10 02:19:00.997935 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:00.997910    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:19:01.054822 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.054771    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dkgv\" (UniqueName: \"kubernetes.io/projected/b37484bc-6764-46ac-9e9f-9a4a6368e355-kube-api-access-8dkgv\") pod \"b37484bc-6764-46ac-9e9f-9a4a6368e355\" (UID: \"b37484bc-6764-46ac-9e9f-9a4a6368e355\") "
Jul 10 02:19:01.054995 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.054887    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b37484bc-6764-46ac-9e9f-9a4a6368e355-extensions-socket-volume\") pod \"b37484bc-6764-46ac-9e9f-9a4a6368e355\" (UID: \"b37484bc-6764-46ac-9e9f-9a4a6368e355\") "
Jul 10 02:19:01.055257 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.055229    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b37484bc-6764-46ac-9e9f-9a4a6368e355-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b37484bc-6764-46ac-9e9f-9a4a6368e355" (UID: "b37484bc-6764-46ac-9e9f-9a4a6368e355"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:19:01.057057 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.057032    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b37484bc-6764-46ac-9e9f-9a4a6368e355-kube-api-access-8dkgv" (OuterVolumeSpecName: "kube-api-access-8dkgv") pod "b37484bc-6764-46ac-9e9f-9a4a6368e355" (UID: "b37484bc-6764-46ac-9e9f-9a4a6368e355"). InnerVolumeSpecName "kube-api-access-8dkgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:19:01.155775 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.155738    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b37484bc-6764-46ac-9e9f-9a4a6368e355-extensions-socket-volume\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:19:01.155775 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.155766    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dkgv\" (UniqueName: \"kubernetes.io/projected/b37484bc-6764-46ac-9e9f-9a4a6368e355-kube-api-access-8dkgv\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:19:01.732322 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.732291    2557 generic.go:356] "Generic (PLEG): container finished" podID="b37484bc-6764-46ac-9e9f-9a4a6368e355" containerID="b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e" exitCode=0
Jul 10 02:19:01.732733 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.732333    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"
Jul 10 02:19:01.732733 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.732342    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2" event={"ID":"b37484bc-6764-46ac-9e9f-9a4a6368e355","Type":"ContainerDied","Data":"b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e"}
Jul 10 02:19:01.732733 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.732384    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2" event={"ID":"b37484bc-6764-46ac-9e9f-9a4a6368e355","Type":"ContainerDied","Data":"cd688d5689fbfbf15e42dcf40af8110a41a87c6226bb10cbf638b15dc98f1855"}
Jul 10 02:19:01.732733 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.732405    2557 scope.go:117] "RemoveContainer" containerID="b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e"
Jul 10 02:19:01.740884 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.740869    2557 scope.go:117] "RemoveContainer" containerID="b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e"
Jul 10 02:19:01.741111 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:19:01.741091    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e\": container with ID starting with b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e not found: ID does not exist" containerID="b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e"
Jul 10 02:19:01.741172 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.741123    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e"} err="failed to get container status \"b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e\": rpc error: code = NotFound desc = could not find container \"b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e\": container with ID starting with b62acbd60b703d462ff4b9f5fba60f9598e5446d6f07d7fd078f384e6344e74e not found: ID does not exist"
Jul 10 02:19:01.754588 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.754566    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"]
Jul 10 02:19:01.765956 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:01.765937    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sltx2"]
Jul 10 02:19:02.241598 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:02.241567    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b37484bc-6764-46ac-9e9f-9a4a6368e355" path="/var/lib/kubelet/pods/b37484bc-6764-46ac-9e9f-9a4a6368e355/volumes"
Jul 10 02:19:17.149863 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.149780    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"]
Jul 10 02:19:17.150287 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.150098    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b37484bc-6764-46ac-9e9f-9a4a6368e355" containerName="manager"
Jul 10 02:19:17.150287 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.150110    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b37484bc-6764-46ac-9e9f-9a4a6368e355" containerName="manager"
Jul 10 02:19:17.150287 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.150188    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b37484bc-6764-46ac-9e9f-9a4a6368e355" containerName="manager"
Jul 10 02:19:17.156492 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.156469    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.158788 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.158753    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-864pm\""
Jul 10 02:19:17.166423 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.166399    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"]
Jul 10 02:19:17.282710 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.282675    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.282710 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.282713    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.282951 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.282733    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.282951 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.282872    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.282951 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.282901    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zz8q\" (UniqueName: \"kubernetes.io/projected/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-kube-api-access-9zz8q\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.282951 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.282927    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.283086 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.282980    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.283086 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.282999    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.283086 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.283047    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384100 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384058    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384100 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384108    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384333 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384128    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384333 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384144    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384333 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384183    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384333 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384202    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zz8q\" (UniqueName: \"kubernetes.io/projected/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-kube-api-access-9zz8q\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384333 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384220    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384333 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384279    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384333 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384306    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384666 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384514    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384666 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384546    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384666 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384626    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.384774 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.384718    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.385099 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.385082    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.386413 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.386395    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.386677 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.386658    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.392227 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.392201    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.392352 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.392235    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zz8q\" (UniqueName: \"kubernetes.io/projected/964c11a5-ebcf-40b5-8ff5-9b8156cc6c24-kube-api-access-9zz8q\") pod \"maas-default-gateway-openshift-default-56654dcf7-xngqk\" (UID: \"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.472748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.472664    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:17.607256 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.607169    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"]
Jul 10 02:19:17.609880 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:19:17.609854    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod964c11a5_ebcf_40b5_8ff5_9b8156cc6c24.slice/crio-07a0606a9d7010f36916b812ee29abb10b714af0239183e563589847c4fb7f24 WatchSource:0}: Error finding container 07a0606a9d7010f36916b812ee29abb10b714af0239183e563589847c4fb7f24: Status 404 returned error can't find the container with id 07a0606a9d7010f36916b812ee29abb10b714af0239183e563589847c4fb7f24
Jul 10 02:19:17.611973 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.611939    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:19:17.612058 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.612025    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:19:17.612103 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.612066    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:19:17.805056 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.805022    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk" event={"ID":"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24","Type":"ContainerStarted","Data":"cd006ade914df6cbd5144211761d05c487df6a9ddfdc631628d330a127cf7519"}
Jul 10 02:19:17.805218 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.805073    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk" event={"ID":"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24","Type":"ContainerStarted","Data":"07a0606a9d7010f36916b812ee29abb10b714af0239183e563589847c4fb7f24"}
Jul 10 02:19:17.825280 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:17.825210    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk" podStartSLOduration=0.825189912 podStartE2EDuration="825.189912ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:17.824196187 +0000 UTC m=+714.140506059" watchObservedRunningTime="2026-07-10 02:19:17.825189912 +0000 UTC m=+714.141499785"
Jul 10 02:19:18.473198 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:18.473161    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:18.477950 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:18.477924    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:18.809951 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:18.809922    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:18.810713 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:18.810693    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:19:18.928656 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:18.928623    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nstmd"]
Jul 10 02:19:18.932001 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:18.931979    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:18.933781 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:18.933762    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 10 02:19:18.933897 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:18.933798    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dc49t\""
Jul 10 02:19:18.940555 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:18.940528    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nstmd"]
Jul 10 02:19:18.998705 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:18.998671    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7bgj\" (UniqueName: \"kubernetes.io/projected/bf047e6e-ff0e-40f9-8717-2b2e157a9414-kube-api-access-g7bgj\") pod \"limitador-limitador-64c94fd64f-nstmd\" (UID: \"bf047e6e-ff0e-40f9-8717-2b2e157a9414\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:18.998887 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:18.998711    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf047e6e-ff0e-40f9-8717-2b2e157a9414-config-file\") pod \"limitador-limitador-64c94fd64f-nstmd\" (UID: \"bf047e6e-ff0e-40f9-8717-2b2e157a9414\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:19.025630 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.025594    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nstmd"]
Jul 10 02:19:19.099543 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.099439    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7bgj\" (UniqueName: \"kubernetes.io/projected/bf047e6e-ff0e-40f9-8717-2b2e157a9414-kube-api-access-g7bgj\") pod \"limitador-limitador-64c94fd64f-nstmd\" (UID: \"bf047e6e-ff0e-40f9-8717-2b2e157a9414\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:19.099543 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.099493    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf047e6e-ff0e-40f9-8717-2b2e157a9414-config-file\") pod \"limitador-limitador-64c94fd64f-nstmd\" (UID: \"bf047e6e-ff0e-40f9-8717-2b2e157a9414\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:19.100186 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.100160    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf047e6e-ff0e-40f9-8717-2b2e157a9414-config-file\") pod \"limitador-limitador-64c94fd64f-nstmd\" (UID: \"bf047e6e-ff0e-40f9-8717-2b2e157a9414\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:19.110235 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.110205    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7bgj\" (UniqueName: \"kubernetes.io/projected/bf047e6e-ff0e-40f9-8717-2b2e157a9414-kube-api-access-g7bgj\") pod \"limitador-limitador-64c94fd64f-nstmd\" (UID: \"bf047e6e-ff0e-40f9-8717-2b2e157a9414\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:19.242465 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.242427    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:19.372061 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.372035    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nstmd"]
Jul 10 02:19:19.374459 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:19:19.374429    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf047e6e_ff0e_40f9_8717_2b2e157a9414.slice/crio-05564ec4d3bd28d9b3b290350d56e8a28be2f6b905b690db8443f51de2ab9da6 WatchSource:0}: Error finding container 05564ec4d3bd28d9b3b290350d56e8a28be2f6b905b690db8443f51de2ab9da6: Status 404 returned error can't find the container with id 05564ec4d3bd28d9b3b290350d56e8a28be2f6b905b690db8443f51de2ab9da6
Jul 10 02:19:19.773392 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.773368    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-6f472"]
Jul 10 02:19:19.777004 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.776983    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6f472"
Jul 10 02:19:19.778696 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.778680    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nwxtt\""
Jul 10 02:19:19.786329 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.786304    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6f472"]
Jul 10 02:19:19.815489 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.815457    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd" event={"ID":"bf047e6e-ff0e-40f9-8717-2b2e157a9414","Type":"ContainerStarted","Data":"05564ec4d3bd28d9b3b290350d56e8a28be2f6b905b690db8443f51de2ab9da6"}
Jul 10 02:19:19.905210 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:19.905174    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r89nn\" (UniqueName: \"kubernetes.io/projected/906940d0-dcbc-4f26-ba95-e984ccec8763-kube-api-access-r89nn\") pod \"authorino-58cd955675-6f472\" (UID: \"906940d0-dcbc-4f26-ba95-e984ccec8763\") " pod="kuadrant-system/authorino-58cd955675-6f472"
Jul 10 02:19:20.005974 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:20.005940    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r89nn\" (UniqueName: \"kubernetes.io/projected/906940d0-dcbc-4f26-ba95-e984ccec8763-kube-api-access-r89nn\") pod \"authorino-58cd955675-6f472\" (UID: \"906940d0-dcbc-4f26-ba95-e984ccec8763\") " pod="kuadrant-system/authorino-58cd955675-6f472"
Jul 10 02:19:20.014941 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:20.014888    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r89nn\" (UniqueName: \"kubernetes.io/projected/906940d0-dcbc-4f26-ba95-e984ccec8763-kube-api-access-r89nn\") pod \"authorino-58cd955675-6f472\" (UID: \"906940d0-dcbc-4f26-ba95-e984ccec8763\") " pod="kuadrant-system/authorino-58cd955675-6f472"
Jul 10 02:19:20.087075 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:20.086984    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6f472"
Jul 10 02:19:20.290423 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:20.290391    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-6f472"]
Jul 10 02:19:20.294228 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:19:20.293471    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod906940d0_dcbc_4f26_ba95_e984ccec8763.slice/crio-e92e48ae739e4f0d6e69b7ad7b3f7c69ce6a81e9e0218eabb221437a998a3bfe WatchSource:0}: Error finding container e92e48ae739e4f0d6e69b7ad7b3f7c69ce6a81e9e0218eabb221437a998a3bfe: Status 404 returned error can't find the container with id e92e48ae739e4f0d6e69b7ad7b3f7c69ce6a81e9e0218eabb221437a998a3bfe
Jul 10 02:19:20.821464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:20.821426    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6f472" event={"ID":"906940d0-dcbc-4f26-ba95-e984ccec8763","Type":"ContainerStarted","Data":"e92e48ae739e4f0d6e69b7ad7b3f7c69ce6a81e9e0218eabb221437a998a3bfe"}
Jul 10 02:19:24.851273 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:24.851241    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd" event={"ID":"bf047e6e-ff0e-40f9-8717-2b2e157a9414","Type":"ContainerStarted","Data":"3e9926b56a248ba81c03c4772d279f83fcdf54780db6ee3d1bd0f5161629cd65"}
Jul 10 02:19:24.851780 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:24.851369    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:24.852786 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:24.852763    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6f472" event={"ID":"906940d0-dcbc-4f26-ba95-e984ccec8763","Type":"ContainerStarted","Data":"012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14"}
Jul 10 02:19:24.878629 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:24.878584    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd" podStartSLOduration=2.133793368 podStartE2EDuration="6.878570344s" totalImagesPullingTime="4.744776976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:24.877070889 +0000 UTC m=+721.193380763" watchObservedRunningTime="2026-07-10 02:19:24.878570344 +0000 UTC m=+721.194880215"
Jul 10 02:19:24.906657 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:24.906608    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-6f472" podStartSLOduration=2.070671432 podStartE2EDuration="5.906594645s" totalImagesPullingTime="3.835923213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:24.905701185 +0000 UTC m=+721.222011058" watchObservedRunningTime="2026-07-10 02:19:24.906594645 +0000 UTC m=+721.222904516"
Jul 10 02:19:25.418604 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:25.418562    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6f472"]
Jul 10 02:19:26.861316 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:26.861276    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-6f472" podUID="906940d0-dcbc-4f26-ba95-e984ccec8763" containerName="authorino" containerID="cri-o://012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14" gracePeriod=30
Jul 10 02:19:27.112144 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.112091    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6f472"
Jul 10 02:19:27.273901 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.273865    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r89nn\" (UniqueName: \"kubernetes.io/projected/906940d0-dcbc-4f26-ba95-e984ccec8763-kube-api-access-r89nn\") pod \"906940d0-dcbc-4f26-ba95-e984ccec8763\" (UID: \"906940d0-dcbc-4f26-ba95-e984ccec8763\") "
Jul 10 02:19:27.276009 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.275975    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906940d0-dcbc-4f26-ba95-e984ccec8763-kube-api-access-r89nn" (OuterVolumeSpecName: "kube-api-access-r89nn") pod "906940d0-dcbc-4f26-ba95-e984ccec8763" (UID: "906940d0-dcbc-4f26-ba95-e984ccec8763"). InnerVolumeSpecName "kube-api-access-r89nn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:19:27.375120 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.375036    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r89nn\" (UniqueName: \"kubernetes.io/projected/906940d0-dcbc-4f26-ba95-e984ccec8763-kube-api-access-r89nn\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:19:27.867346 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.867315    2557 generic.go:356] "Generic (PLEG): container finished" podID="906940d0-dcbc-4f26-ba95-e984ccec8763" containerID="012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14" exitCode=0
Jul 10 02:19:27.867781 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.867362    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-6f472"
Jul 10 02:19:27.867781 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.867384    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6f472" event={"ID":"906940d0-dcbc-4f26-ba95-e984ccec8763","Type":"ContainerDied","Data":"012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14"}
Jul 10 02:19:27.867781 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.867414    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-6f472" event={"ID":"906940d0-dcbc-4f26-ba95-e984ccec8763","Type":"ContainerDied","Data":"e92e48ae739e4f0d6e69b7ad7b3f7c69ce6a81e9e0218eabb221437a998a3bfe"}
Jul 10 02:19:27.867781 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.867433    2557 scope.go:117] "RemoveContainer" containerID="012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14"
Jul 10 02:19:27.875857 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.875838    2557 scope.go:117] "RemoveContainer" containerID="012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14"
Jul 10 02:19:27.876112 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:19:27.876096    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14\": container with ID starting with 012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14 not found: ID does not exist" containerID="012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14"
Jul 10 02:19:27.876156 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.876119    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14"} err="failed to get container status \"012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14\": rpc error: code = NotFound desc = could not find container \"012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14\": container with ID starting with 012d89adaf80a0f6e6e39a8ef4b34ce21e108e9e9c687b445a4d078803475d14 not found: ID does not exist"
Jul 10 02:19:27.888000 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.887973    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-6f472"]
Jul 10 02:19:27.893779 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:27.893757    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-6f472"]
Jul 10 02:19:28.241387 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:28.241304    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="906940d0-dcbc-4f26-ba95-e984ccec8763" path="/var/lib/kubelet/pods/906940d0-dcbc-4f26-ba95-e984ccec8763/volumes"
Jul 10 02:19:35.858381 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:35.858352    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:36.417512 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:36.417483    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nstmd"]
Jul 10 02:19:36.417706 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:36.417666    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd" podUID="bf047e6e-ff0e-40f9-8717-2b2e157a9414" containerName="limitador" containerID="cri-o://3e9926b56a248ba81c03c4772d279f83fcdf54780db6ee3d1bd0f5161629cd65" gracePeriod=30
Jul 10 02:19:36.910517 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:36.910486    2557 generic.go:356] "Generic (PLEG): container finished" podID="bf047e6e-ff0e-40f9-8717-2b2e157a9414" containerID="3e9926b56a248ba81c03c4772d279f83fcdf54780db6ee3d1bd0f5161629cd65" exitCode=0
Jul 10 02:19:36.910916 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:36.910562    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd" event={"ID":"bf047e6e-ff0e-40f9-8717-2b2e157a9414","Type":"ContainerDied","Data":"3e9926b56a248ba81c03c4772d279f83fcdf54780db6ee3d1bd0f5161629cd65"}
Jul 10 02:19:37.366686 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.366665    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:37.463445 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.463403    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf047e6e-ff0e-40f9-8717-2b2e157a9414-config-file\") pod \"bf047e6e-ff0e-40f9-8717-2b2e157a9414\" (UID: \"bf047e6e-ff0e-40f9-8717-2b2e157a9414\") "
Jul 10 02:19:37.463445 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.463450    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7bgj\" (UniqueName: \"kubernetes.io/projected/bf047e6e-ff0e-40f9-8717-2b2e157a9414-kube-api-access-g7bgj\") pod \"bf047e6e-ff0e-40f9-8717-2b2e157a9414\" (UID: \"bf047e6e-ff0e-40f9-8717-2b2e157a9414\") "
Jul 10 02:19:37.463766 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.463740    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf047e6e-ff0e-40f9-8717-2b2e157a9414-config-file" (OuterVolumeSpecName: "config-file") pod "bf047e6e-ff0e-40f9-8717-2b2e157a9414" (UID: "bf047e6e-ff0e-40f9-8717-2b2e157a9414"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:19:37.465520 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.465501    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf047e6e-ff0e-40f9-8717-2b2e157a9414-kube-api-access-g7bgj" (OuterVolumeSpecName: "kube-api-access-g7bgj") pod "bf047e6e-ff0e-40f9-8717-2b2e157a9414" (UID: "bf047e6e-ff0e-40f9-8717-2b2e157a9414"). InnerVolumeSpecName "kube-api-access-g7bgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:19:37.564185 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.564156    2557 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/bf047e6e-ff0e-40f9-8717-2b2e157a9414-config-file\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:19:37.564185 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.564185    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7bgj\" (UniqueName: \"kubernetes.io/projected/bf047e6e-ff0e-40f9-8717-2b2e157a9414-kube-api-access-g7bgj\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:19:37.916013 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.915934    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd" event={"ID":"bf047e6e-ff0e-40f9-8717-2b2e157a9414","Type":"ContainerDied","Data":"05564ec4d3bd28d9b3b290350d56e8a28be2f6b905b690db8443f51de2ab9da6"}
Jul 10 02:19:37.916013 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.915961    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-nstmd"
Jul 10 02:19:37.916013 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.915998    2557 scope.go:117] "RemoveContainer" containerID="3e9926b56a248ba81c03c4772d279f83fcdf54780db6ee3d1bd0f5161629cd65"
Jul 10 02:19:37.936647 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.936625    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nstmd"]
Jul 10 02:19:37.943772 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:37.943752    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-nstmd"]
Jul 10 02:19:38.241034 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:38.240958    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf047e6e-ff0e-40f9-8717-2b2e157a9414" path="/var/lib/kubelet/pods/bf047e6e-ff0e-40f9-8717-2b2e157a9414/volumes"
Jul 10 02:19:40.707491 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.707453    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"]
Jul 10 02:19:40.708281 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.708259    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf047e6e-ff0e-40f9-8717-2b2e157a9414" containerName="limitador"
Jul 10 02:19:40.708281 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.708283    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf047e6e-ff0e-40f9-8717-2b2e157a9414" containerName="limitador"
Jul 10 02:19:40.708433 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.708309    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906940d0-dcbc-4f26-ba95-e984ccec8763" containerName="authorino"
Jul 10 02:19:40.708433 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.708317    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="906940d0-dcbc-4f26-ba95-e984ccec8763" containerName="authorino"
Jul 10 02:19:40.708433 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.708394    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="906940d0-dcbc-4f26-ba95-e984ccec8763" containerName="authorino"
Jul 10 02:19:40.708433 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.708406    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf047e6e-ff0e-40f9-8717-2b2e157a9414" containerName="limitador"
Jul 10 02:19:40.712706 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.712684    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"
Jul 10 02:19:40.714928 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.714905    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 10 02:19:40.715160 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.715140    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-mlmqh\""
Jul 10 02:19:40.715160 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.715147    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 10 02:19:40.715330 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.715223    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 10 02:19:40.716203 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.716185    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"]
Jul 10 02:19:40.892633 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.892590    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjqlt\" (UniqueName: \"kubernetes.io/projected/b1d34833-be71-4e9f-8873-93a8aee9baf3-kube-api-access-vjqlt\") pod \"postgres-69fb4594cf-sz4gr\" (UID: \"b1d34833-be71-4e9f-8873-93a8aee9baf3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"
Jul 10 02:19:40.892836 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.892644    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b1d34833-be71-4e9f-8873-93a8aee9baf3-data\") pod \"postgres-69fb4594cf-sz4gr\" (UID: \"b1d34833-be71-4e9f-8873-93a8aee9baf3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"
Jul 10 02:19:40.993911 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.993828    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b1d34833-be71-4e9f-8873-93a8aee9baf3-data\") pod \"postgres-69fb4594cf-sz4gr\" (UID: \"b1d34833-be71-4e9f-8873-93a8aee9baf3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"
Jul 10 02:19:40.994056 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.993915    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjqlt\" (UniqueName: \"kubernetes.io/projected/b1d34833-be71-4e9f-8873-93a8aee9baf3-kube-api-access-vjqlt\") pod \"postgres-69fb4594cf-sz4gr\" (UID: \"b1d34833-be71-4e9f-8873-93a8aee9baf3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"
Jul 10 02:19:40.994199 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:40.994178    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b1d34833-be71-4e9f-8873-93a8aee9baf3-data\") pod \"postgres-69fb4594cf-sz4gr\" (UID: \"b1d34833-be71-4e9f-8873-93a8aee9baf3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"
Jul 10 02:19:41.002656 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:41.002631    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjqlt\" (UniqueName: \"kubernetes.io/projected/b1d34833-be71-4e9f-8873-93a8aee9baf3-kube-api-access-vjqlt\") pod \"postgres-69fb4594cf-sz4gr\" (UID: \"b1d34833-be71-4e9f-8873-93a8aee9baf3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"
Jul 10 02:19:41.023492 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:41.023466    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"
Jul 10 02:19:41.151022 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:41.151002    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"]
Jul 10 02:19:41.153052 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:19:41.153020    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d34833_be71_4e9f_8873_93a8aee9baf3.slice/crio-8e215652aed780d197c56e175efeab1b7d7d713fa0b7efede573217bc7d6132a WatchSource:0}: Error finding container 8e215652aed780d197c56e175efeab1b7d7d713fa0b7efede573217bc7d6132a: Status 404 returned error can't find the container with id 8e215652aed780d197c56e175efeab1b7d7d713fa0b7efede573217bc7d6132a
Jul 10 02:19:41.934536 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:41.934503    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr" event={"ID":"b1d34833-be71-4e9f-8873-93a8aee9baf3","Type":"ContainerStarted","Data":"8e215652aed780d197c56e175efeab1b7d7d713fa0b7efede573217bc7d6132a"}
Jul 10 02:19:46.960318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:46.960278    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr" event={"ID":"b1d34833-be71-4e9f-8873-93a8aee9baf3","Type":"ContainerStarted","Data":"04e838c4d730972354aca52eef8372bc1a49f6cfa92d105d276e3860d612459b"}
Jul 10 02:19:46.960773 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:46.960345    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"
Jul 10 02:19:46.977545 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:46.977491    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr" podStartSLOduration=2.007001161 podStartE2EDuration="6.977475111s" totalImagesPullingTime="4.97047395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:46.97539923 +0000 UTC m=+743.291709102" watchObservedRunningTime="2026-07-10 02:19:46.977475111 +0000 UTC m=+743.293784983"
Jul 10 02:19:52.993939 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:52.993910    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sz4gr"
Jul 10 02:19:53.573958 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.573923    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rlrq8"]
Jul 10 02:19:53.577246 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.577230    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rlrq8"
Jul 10 02:19:53.578980 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.578958    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-nwxtt\""
Jul 10 02:19:53.586237 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.586217    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rlrq8"]
Jul 10 02:19:53.602737 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.602668    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgmrv\" (UniqueName: \"kubernetes.io/projected/9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d-kube-api-access-bgmrv\") pod \"authorino-5d4b67c446-rlrq8\" (UID: \"9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d\") " pod="kuadrant-system/authorino-5d4b67c446-rlrq8"
Jul 10 02:19:53.703282 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.703249    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgmrv\" (UniqueName: \"kubernetes.io/projected/9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d-kube-api-access-bgmrv\") pod \"authorino-5d4b67c446-rlrq8\" (UID: \"9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d\") " pod="kuadrant-system/authorino-5d4b67c446-rlrq8"
Jul 10 02:19:53.712670 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.712638    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgmrv\" (UniqueName: \"kubernetes.io/projected/9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d-kube-api-access-bgmrv\") pod \"authorino-5d4b67c446-rlrq8\" (UID: \"9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d\") " pod="kuadrant-system/authorino-5d4b67c446-rlrq8"
Jul 10 02:19:53.824228 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.824138    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rlrq8"]
Jul 10 02:19:53.824486 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.824468    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rlrq8"
Jul 10 02:19:53.852104 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.852074    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b569ccc47-bj7f9"]
Jul 10 02:19:53.855567 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.855546    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b569ccc47-bj7f9"
Jul 10 02:19:53.863279 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.863162    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b569ccc47-bj7f9"]
Jul 10 02:19:53.904959 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.904896    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47mws\" (UniqueName: \"kubernetes.io/projected/d5fe6463-c20c-4a19-b2bd-0b1af3ab4654-kube-api-access-47mws\") pod \"authorino-7b569ccc47-bj7f9\" (UID: \"d5fe6463-c20c-4a19-b2bd-0b1af3ab4654\") " pod="kuadrant-system/authorino-7b569ccc47-bj7f9"
Jul 10 02:19:53.955463 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.955350    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rlrq8"]
Jul 10 02:19:53.958171 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:19:53.958143    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a7d56dd_f6b0_4b1f_a3a9_236de8a2b53d.slice/crio-884e92ad4ca44141ffaf51451c0af7b7d01c21904a4885e7a492bc4b270853d3 WatchSource:0}: Error finding container 884e92ad4ca44141ffaf51451c0af7b7d01c21904a4885e7a492bc4b270853d3: Status 404 returned error can't find the container with id 884e92ad4ca44141ffaf51451c0af7b7d01c21904a4885e7a492bc4b270853d3
Jul 10 02:19:53.991792 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:53.991759    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rlrq8" event={"ID":"9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d","Type":"ContainerStarted","Data":"884e92ad4ca44141ffaf51451c0af7b7d01c21904a4885e7a492bc4b270853d3"}
Jul 10 02:19:54.006114 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.006091    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47mws\" (UniqueName: \"kubernetes.io/projected/d5fe6463-c20c-4a19-b2bd-0b1af3ab4654-kube-api-access-47mws\") pod \"authorino-7b569ccc47-bj7f9\" (UID: \"d5fe6463-c20c-4a19-b2bd-0b1af3ab4654\") " pod="kuadrant-system/authorino-7b569ccc47-bj7f9"
Jul 10 02:19:54.016998 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.016972    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47mws\" (UniqueName: \"kubernetes.io/projected/d5fe6463-c20c-4a19-b2bd-0b1af3ab4654-kube-api-access-47mws\") pod \"authorino-7b569ccc47-bj7f9\" (UID: \"d5fe6463-c20c-4a19-b2bd-0b1af3ab4654\") " pod="kuadrant-system/authorino-7b569ccc47-bj7f9"
Jul 10 02:19:54.145339 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.145260    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b569ccc47-bj7f9"]
Jul 10 02:19:54.145505 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.145493    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b569ccc47-bj7f9"
Jul 10 02:19:54.177149 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.177113    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56b64b8547-zvwp7"]
Jul 10 02:19:54.181107 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.181086    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b64b8547-zvwp7"
Jul 10 02:19:54.182836 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.182792    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 10 02:19:54.187206 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.187183    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56b64b8547-zvwp7"]
Jul 10 02:19:54.207222 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.207183    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-tls-cert\") pod \"authorino-56b64b8547-zvwp7\" (UID: \"b85a3937-40ab-4db7-8ea1-fcd2303e5e72\") " pod="kuadrant-system/authorino-56b64b8547-zvwp7"
Jul 10 02:19:54.207343 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.207234    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzl4v\" (UniqueName: \"kubernetes.io/projected/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-kube-api-access-bzl4v\") pod \"authorino-56b64b8547-zvwp7\" (UID: \"b85a3937-40ab-4db7-8ea1-fcd2303e5e72\") " pod="kuadrant-system/authorino-56b64b8547-zvwp7"
Jul 10 02:19:54.279572 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.279456    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b569ccc47-bj7f9"]
Jul 10 02:19:54.282001 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:19:54.281972    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5fe6463_c20c_4a19_b2bd_0b1af3ab4654.slice/crio-b85d2e8a380591abbed2718390620e74d868ddabd7c9f6744830999b33e61d56 WatchSource:0}: Error finding container b85d2e8a380591abbed2718390620e74d868ddabd7c9f6744830999b33e61d56: Status 404 returned error can't find the container with id b85d2e8a380591abbed2718390620e74d868ddabd7c9f6744830999b33e61d56
Jul 10 02:19:54.307758 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.307734    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-tls-cert\") pod \"authorino-56b64b8547-zvwp7\" (UID: \"b85a3937-40ab-4db7-8ea1-fcd2303e5e72\") " pod="kuadrant-system/authorino-56b64b8547-zvwp7"
Jul 10 02:19:54.307898 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.307767    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzl4v\" (UniqueName: \"kubernetes.io/projected/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-kube-api-access-bzl4v\") pod \"authorino-56b64b8547-zvwp7\" (UID: \"b85a3937-40ab-4db7-8ea1-fcd2303e5e72\") " pod="kuadrant-system/authorino-56b64b8547-zvwp7"
Jul 10 02:19:54.309990 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.309970    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-tls-cert\") pod \"authorino-56b64b8547-zvwp7\" (UID: \"b85a3937-40ab-4db7-8ea1-fcd2303e5e72\") " pod="kuadrant-system/authorino-56b64b8547-zvwp7"
Jul 10 02:19:54.316030 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.316003    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzl4v\" (UniqueName: \"kubernetes.io/projected/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-kube-api-access-bzl4v\") pod \"authorino-56b64b8547-zvwp7\" (UID: \"b85a3937-40ab-4db7-8ea1-fcd2303e5e72\") " pod="kuadrant-system/authorino-56b64b8547-zvwp7"
Jul 10 02:19:54.493083 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.493050    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b64b8547-zvwp7"
Jul 10 02:19:54.626097 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.626065    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56b64b8547-zvwp7"]
Jul 10 02:19:54.629144 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:19:54.629114    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb85a3937_40ab_4db7_8ea1_fcd2303e5e72.slice/crio-e2222aeb28720afae7dc44f0373b4ccca6d427a03d0443cc72bf58fa2a4cb850 WatchSource:0}: Error finding container e2222aeb28720afae7dc44f0373b4ccca6d427a03d0443cc72bf58fa2a4cb850: Status 404 returned error can't find the container with id e2222aeb28720afae7dc44f0373b4ccca6d427a03d0443cc72bf58fa2a4cb850
Jul 10 02:19:54.998052 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.997946    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rlrq8" event={"ID":"9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d","Type":"ContainerStarted","Data":"3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d"}
Jul 10 02:19:54.998052 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.997971    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-rlrq8" podUID="9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d" containerName="authorino" containerID="cri-o://3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d" gracePeriod=30
Jul 10 02:19:54.999165 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:54.999145    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56b64b8547-zvwp7" event={"ID":"b85a3937-40ab-4db7-8ea1-fcd2303e5e72","Type":"ContainerStarted","Data":"e2222aeb28720afae7dc44f0373b4ccca6d427a03d0443cc72bf58fa2a4cb850"}
Jul 10 02:19:55.000580 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.000560    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b569ccc47-bj7f9" event={"ID":"d5fe6463-c20c-4a19-b2bd-0b1af3ab4654","Type":"ContainerStarted","Data":"1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140"}
Jul 10 02:19:55.000652 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.000589    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b569ccc47-bj7f9" event={"ID":"d5fe6463-c20c-4a19-b2bd-0b1af3ab4654","Type":"ContainerStarted","Data":"b85d2e8a380591abbed2718390620e74d868ddabd7c9f6744830999b33e61d56"}
Jul 10 02:19:55.000652 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.000621    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7b569ccc47-bj7f9" podUID="d5fe6463-c20c-4a19-b2bd-0b1af3ab4654" containerName="authorino" containerID="cri-o://1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140" gracePeriod=30
Jul 10 02:19:55.012886 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.012841    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-rlrq8" podStartSLOduration=1.589022406 podStartE2EDuration="2.012824959s" totalImagesPullingTime="423.802553ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:55.011600352 +0000 UTC m=+751.327910223" watchObservedRunningTime="2026-07-10 02:19:55.012824959 +0000 UTC m=+751.329134822"
Jul 10 02:19:55.027486 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.027440    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b569ccc47-bj7f9" podStartSLOduration=1.600921719 podStartE2EDuration="2.027422902s" totalImagesPullingTime="426.501183ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:55.025596675 +0000 UTC m=+751.341906547" watchObservedRunningTime="2026-07-10 02:19:55.027422902 +0000 UTC m=+751.343732774"
Jul 10 02:19:55.253976 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.253915    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rlrq8"
Jul 10 02:19:55.282412 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.282390    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b569ccc47-bj7f9"
Jul 10 02:19:55.315966 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.315940    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgmrv\" (UniqueName: \"kubernetes.io/projected/9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d-kube-api-access-bgmrv\") pod \"9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d\" (UID: \"9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d\") "
Jul 10 02:19:55.316144 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.315996    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47mws\" (UniqueName: \"kubernetes.io/projected/d5fe6463-c20c-4a19-b2bd-0b1af3ab4654-kube-api-access-47mws\") pod \"d5fe6463-c20c-4a19-b2bd-0b1af3ab4654\" (UID: \"d5fe6463-c20c-4a19-b2bd-0b1af3ab4654\") "
Jul 10 02:19:55.318100 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.318070    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d-kube-api-access-bgmrv" (OuterVolumeSpecName: "kube-api-access-bgmrv") pod "9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d" (UID: "9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d"). InnerVolumeSpecName "kube-api-access-bgmrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:19:55.318100 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.318101    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5fe6463-c20c-4a19-b2bd-0b1af3ab4654-kube-api-access-47mws" (OuterVolumeSpecName: "kube-api-access-47mws") pod "d5fe6463-c20c-4a19-b2bd-0b1af3ab4654" (UID: "d5fe6463-c20c-4a19-b2bd-0b1af3ab4654"). InnerVolumeSpecName "kube-api-access-47mws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:19:55.416720 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.416691    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgmrv\" (UniqueName: \"kubernetes.io/projected/9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d-kube-api-access-bgmrv\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:19:55.416720 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:55.416717    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47mws\" (UniqueName: \"kubernetes.io/projected/d5fe6463-c20c-4a19-b2bd-0b1af3ab4654-kube-api-access-47mws\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:19:56.007084 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.006989    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56b64b8547-zvwp7" event={"ID":"b85a3937-40ab-4db7-8ea1-fcd2303e5e72","Type":"ContainerStarted","Data":"f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3"}
Jul 10 02:19:56.008153 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.008127    2557 generic.go:356] "Generic (PLEG): container finished" podID="d5fe6463-c20c-4a19-b2bd-0b1af3ab4654" containerID="1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140" exitCode=0
Jul 10 02:19:56.008276 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.008167    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b569ccc47-bj7f9"
Jul 10 02:19:56.008276 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.008210    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b569ccc47-bj7f9" event={"ID":"d5fe6463-c20c-4a19-b2bd-0b1af3ab4654","Type":"ContainerDied","Data":"1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140"}
Jul 10 02:19:56.008276 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.008249    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b569ccc47-bj7f9" event={"ID":"d5fe6463-c20c-4a19-b2bd-0b1af3ab4654","Type":"ContainerDied","Data":"b85d2e8a380591abbed2718390620e74d868ddabd7c9f6744830999b33e61d56"}
Jul 10 02:19:56.008276 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.008267    2557 scope.go:117] "RemoveContainer" containerID="1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140"
Jul 10 02:19:56.009508 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.009379    2557 generic.go:356] "Generic (PLEG): container finished" podID="9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d" containerID="3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d" exitCode=0
Jul 10 02:19:56.009508 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.009411    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rlrq8" event={"ID":"9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d","Type":"ContainerDied","Data":"3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d"}
Jul 10 02:19:56.009508 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.009430    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-rlrq8" event={"ID":"9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d","Type":"ContainerDied","Data":"884e92ad4ca44141ffaf51451c0af7b7d01c21904a4885e7a492bc4b270853d3"}
Jul 10 02:19:56.009508 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.009429    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-rlrq8"
Jul 10 02:19:56.017378 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.017110    2557 scope.go:117] "RemoveContainer" containerID="1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140"
Jul 10 02:19:56.017671 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:19:56.017479    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140\": container with ID starting with 1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140 not found: ID does not exist" containerID="1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140"
Jul 10 02:19:56.017671 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.017507    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140"} err="failed to get container status \"1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140\": rpc error: code = NotFound desc = could not find container \"1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140\": container with ID starting with 1af1e29357db6bf3ad51a2413a27b3da0e6344bba212454ee2043ac6729d3140 not found: ID does not exist"
Jul 10 02:19:56.017671 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.017532    2557 scope.go:117] "RemoveContainer" containerID="3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d"
Jul 10 02:19:56.023763 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.023716    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56b64b8547-zvwp7" podStartSLOduration=1.593395486 podStartE2EDuration="2.023698799s" totalImagesPullingTime="430.303313ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:19:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:19:56.021108414 +0000 UTC m=+752.337418283" watchObservedRunningTime="2026-07-10 02:19:56.023698799 +0000 UTC m=+752.340008672"
Jul 10 02:19:56.026076 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.026052    2557 scope.go:117] "RemoveContainer" containerID="3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d"
Jul 10 02:19:56.026348 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:19:56.026331    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d\": container with ID starting with 3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d not found: ID does not exist" containerID="3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d"
Jul 10 02:19:56.026402 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.026353    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d"} err="failed to get container status \"3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d\": rpc error: code = NotFound desc = could not find container \"3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d\": container with ID starting with 3132b1741e1d8604bfa893e3b1d6414667ac4407ca3576e61c8742624d3bab8d not found: ID does not exist"
Jul 10 02:19:56.037821 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.037779    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rlrq8"]
Jul 10 02:19:56.043064 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.043041    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-rlrq8"]
Jul 10 02:19:56.054249 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.054225    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7b569ccc47-bj7f9"]
Jul 10 02:19:56.060689 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.060663    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7b569ccc47-bj7f9"]
Jul 10 02:19:56.241999 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.241970    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d" path="/var/lib/kubelet/pods/9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d/volumes"
Jul 10 02:19:56.242295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:19:56.242283    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5fe6463-c20c-4a19-b2bd-0b1af3ab4654" path="/var/lib/kubelet/pods/d5fe6463-c20c-4a19-b2bd-0b1af3ab4654/volumes"
Jul 10 02:20:00.502893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.502854    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65b89ddcc6-7qk9w"]
Jul 10 02:20:00.503597 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.503574    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5fe6463-c20c-4a19-b2bd-0b1af3ab4654" containerName="authorino"
Jul 10 02:20:00.503744 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.503731    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5fe6463-c20c-4a19-b2bd-0b1af3ab4654" containerName="authorino"
Jul 10 02:20:00.503873 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.503861    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d" containerName="authorino"
Jul 10 02:20:00.503959 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.503950    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d" containerName="authorino"
Jul 10 02:20:00.504200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.504187    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a7d56dd-f6b0-4b1f-a3a9-236de8a2b53d" containerName="authorino"
Jul 10 02:20:00.504287 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.504278    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5fe6463-c20c-4a19-b2bd-0b1af3ab4654" containerName="authorino"
Jul 10 02:20:00.509423 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.509400    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:00.511160 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.511137    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 10 02:20:00.511269 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.511177    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 10 02:20:00.511269 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.511184    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9dx8t\""
Jul 10 02:20:00.515659 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.515635    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65b89ddcc6-7qk9w"]
Jul 10 02:20:00.558688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.558647    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf99v\" (UniqueName: \"kubernetes.io/projected/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-kube-api-access-jf99v\") pod \"maas-controller-65b89ddcc6-7qk9w\" (UID: \"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4\") " pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:00.558874 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.558711    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-webhook-cert\") pod \"maas-controller-65b89ddcc6-7qk9w\" (UID: \"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4\") " pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:00.659235 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.659200    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf99v\" (UniqueName: \"kubernetes.io/projected/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-kube-api-access-jf99v\") pod \"maas-controller-65b89ddcc6-7qk9w\" (UID: \"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4\") " pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:00.659420 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.659251    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-webhook-cert\") pod \"maas-controller-65b89ddcc6-7qk9w\" (UID: \"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4\") " pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:00.659420 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:20:00.659360    2557 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 10 02:20:00.659501 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:20:00.659424    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-webhook-cert podName:2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4 nodeName:}" failed. No retries permitted until 2026-07-10 02:20:01.159406463 +0000 UTC m=+757.475716314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-webhook-cert") pod "maas-controller-65b89ddcc6-7qk9w" (UID: "2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4") : secret "maas-controller-webhook-cert" not found
Jul 10 02:20:00.669789 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.669762    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf99v\" (UniqueName: \"kubernetes.io/projected/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-kube-api-access-jf99v\") pod \"maas-controller-65b89ddcc6-7qk9w\" (UID: \"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4\") " pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:00.673970 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.673947    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56f677784f-65p67"]
Jul 10 02:20:00.677314 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.677296    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:20:00.687728 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.687706    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56f677784f-65p67"]
Jul 10 02:20:00.760652 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.760569    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n66rg\" (UniqueName: \"kubernetes.io/projected/b308d089-1156-4914-a05a-477037a69c28-kube-api-access-n66rg\") pod \"maas-controller-56f677784f-65p67\" (UID: \"b308d089-1156-4914-a05a-477037a69c28\") " pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:20:00.760652 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.760615    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b308d089-1156-4914-a05a-477037a69c28-webhook-cert\") pod \"maas-controller-56f677784f-65p67\" (UID: \"b308d089-1156-4914-a05a-477037a69c28\") " pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:20:00.861371 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.861339    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n66rg\" (UniqueName: \"kubernetes.io/projected/b308d089-1156-4914-a05a-477037a69c28-kube-api-access-n66rg\") pod \"maas-controller-56f677784f-65p67\" (UID: \"b308d089-1156-4914-a05a-477037a69c28\") " pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:20:00.861476 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.861385    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b308d089-1156-4914-a05a-477037a69c28-webhook-cert\") pod \"maas-controller-56f677784f-65p67\" (UID: \"b308d089-1156-4914-a05a-477037a69c28\") " pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:20:00.863723 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.863694    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b308d089-1156-4914-a05a-477037a69c28-webhook-cert\") pod \"maas-controller-56f677784f-65p67\" (UID: \"b308d089-1156-4914-a05a-477037a69c28\") " pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:20:00.872159 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.872138    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n66rg\" (UniqueName: \"kubernetes.io/projected/b308d089-1156-4914-a05a-477037a69c28-kube-api-access-n66rg\") pod \"maas-controller-56f677784f-65p67\" (UID: \"b308d089-1156-4914-a05a-477037a69c28\") " pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:20:00.987168 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:00.987131    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:20:01.115757 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:01.115726    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56f677784f-65p67"]
Jul 10 02:20:01.119525 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:20:01.119499    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb308d089_1156_4914_a05a_477037a69c28.slice/crio-1b0e19aa45185a40a32c6039ca552a9356ff61e1d819b67fd4a1c4e735d3f352 WatchSource:0}: Error finding container 1b0e19aa45185a40a32c6039ca552a9356ff61e1d819b67fd4a1c4e735d3f352: Status 404 returned error can't find the container with id 1b0e19aa45185a40a32c6039ca552a9356ff61e1d819b67fd4a1c4e735d3f352
Jul 10 02:20:01.164430 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:01.164396    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-webhook-cert\") pod \"maas-controller-65b89ddcc6-7qk9w\" (UID: \"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4\") " pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:01.166819 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:01.166790    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-webhook-cert\") pod \"maas-controller-65b89ddcc6-7qk9w\" (UID: \"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4\") " pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:01.419914 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:01.419827    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:01.554281 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:01.554081    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65b89ddcc6-7qk9w"]
Jul 10 02:20:01.556422 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:20:01.556398    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ddfd6d1_3fd6_4902_8768_f7419a2b3fb4.slice/crio-5383304e81b798d1039ebe223309e3f1e5247c42b9174ab7e53bc4700c45257c WatchSource:0}: Error finding container 5383304e81b798d1039ebe223309e3f1e5247c42b9174ab7e53bc4700c45257c: Status 404 returned error can't find the container with id 5383304e81b798d1039ebe223309e3f1e5247c42b9174ab7e53bc4700c45257c
Jul 10 02:20:02.040186 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:02.040143    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56f677784f-65p67" event={"ID":"b308d089-1156-4914-a05a-477037a69c28","Type":"ContainerStarted","Data":"1b0e19aa45185a40a32c6039ca552a9356ff61e1d819b67fd4a1c4e735d3f352"}
Jul 10 02:20:02.041875 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:02.041841    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w" event={"ID":"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4","Type":"ContainerStarted","Data":"5383304e81b798d1039ebe223309e3f1e5247c42b9174ab7e53bc4700c45257c"}
Jul 10 02:20:05.059537 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:05.059496    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w" event={"ID":"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4","Type":"ContainerStarted","Data":"aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236"}
Jul 10 02:20:05.060044 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:05.059601    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:05.060949 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:05.060926    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56f677784f-65p67" event={"ID":"b308d089-1156-4914-a05a-477037a69c28","Type":"ContainerStarted","Data":"c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235"}
Jul 10 02:20:05.061060 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:05.061052    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:20:05.078453 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:05.078400    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w" podStartSLOduration=2.43280531 podStartE2EDuration="5.078387052s" totalImagesPullingTime="2.645581742s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:20:05.076888273 +0000 UTC m=+761.393198166" watchObservedRunningTime="2026-07-10 02:20:05.078387052 +0000 UTC m=+761.394696923"
Jul 10 02:20:05.101107 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:05.101054    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56f677784f-65p67" podStartSLOduration=2.029120894 podStartE2EDuration="5.101040046s" totalImagesPullingTime="3.071919152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:20:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:20:05.098451226 +0000 UTC m=+761.414761110" watchObservedRunningTime="2026-07-10 02:20:05.101040046 +0000 UTC m=+761.417349918"
Jul 10 02:20:16.068519 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:16.068487    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:16.068900 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:16.068556    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:20:16.148131 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:16.148102    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65b89ddcc6-7qk9w"]
Jul 10 02:20:16.148351 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:16.148311    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w" podUID="2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4" containerName="manager" containerID="cri-o://aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236" gracePeriod=30
Jul 10 02:20:16.400578 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:16.400548    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:16.498320 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:16.498280    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-webhook-cert\") pod \"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4\" (UID: \"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4\") "
Jul 10 02:20:16.498320 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:16.498327    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf99v\" (UniqueName: \"kubernetes.io/projected/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-kube-api-access-jf99v\") pod \"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4\" (UID: \"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4\") "
Jul 10 02:20:16.500588 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:16.500557    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-kube-api-access-jf99v" (OuterVolumeSpecName: "kube-api-access-jf99v") pod "2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4" (UID: "2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4"). InnerVolumeSpecName "kube-api-access-jf99v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:20:16.500702 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:16.500587    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4" (UID: "2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:20:16.599663 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:16.599628    2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-webhook-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:20:16.599663 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:16.599659    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf99v\" (UniqueName: \"kubernetes.io/projected/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4-kube-api-access-jf99v\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:20:17.116584 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:17.116549    2557 generic.go:356] "Generic (PLEG): container finished" podID="2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4" containerID="aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236" exitCode=0
Jul 10 02:20:17.117086 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:17.116598    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w"
Jul 10 02:20:17.117086 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:17.116619    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w" event={"ID":"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4","Type":"ContainerDied","Data":"aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236"}
Jul 10 02:20:17.117086 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:17.116660    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65b89ddcc6-7qk9w" event={"ID":"2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4","Type":"ContainerDied","Data":"5383304e81b798d1039ebe223309e3f1e5247c42b9174ab7e53bc4700c45257c"}
Jul 10 02:20:17.117086 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:17.116682    2557 scope.go:117] "RemoveContainer" containerID="aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236"
Jul 10 02:20:17.125285 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:17.125267    2557 scope.go:117] "RemoveContainer" containerID="aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236"
Jul 10 02:20:17.125564 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:20:17.125547    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236\": container with ID starting with aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236 not found: ID does not exist" containerID="aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236"
Jul 10 02:20:17.125611 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:17.125573    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236"} err="failed to get container status \"aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236\": rpc error: code = NotFound desc = could not find container \"aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236\": container with ID starting with aab40211008d58e6abb3ce6a33faefd9a181f0d3f2a1aec34d3e2464f4b91236 not found: ID does not exist"
Jul 10 02:20:17.139637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:17.139616    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65b89ddcc6-7qk9w"]
Jul 10 02:20:17.144442 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:17.144421    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65b89ddcc6-7qk9w"]
Jul 10 02:20:18.241315 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:18.241281    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4" path="/var/lib/kubelet/pods/2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4/volumes"
Jul 10 02:20:39.627815 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.627778    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-654967854b-8p7lc"]
Jul 10 02:20:39.628281 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.628113    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4" containerName="manager"
Jul 10 02:20:39.628281 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.628124    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4" containerName="manager"
Jul 10 02:20:39.628281 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.628210    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ddfd6d1-3fd6-4902-8768-f7419a2b3fb4" containerName="manager"
Jul 10 02:20:39.634392 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.634373    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:39.636623 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.636600    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 10 02:20:39.636767 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.636604    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-r7s9m\""
Jul 10 02:20:39.644002 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.643975    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-654967854b-8p7lc"]
Jul 10 02:20:39.688663 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.688630    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c1df2d1-4327-40cf-815c-e0c07946e15f-maas-api-tls\") pod \"maas-api-654967854b-8p7lc\" (UID: \"1c1df2d1-4327-40cf-815c-e0c07946e15f\") " pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:39.688852 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.688676    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c1df2d1-4327-40cf-815c-e0c07946e15f-openshift-service-ca\") pod \"maas-api-654967854b-8p7lc\" (UID: \"1c1df2d1-4327-40cf-815c-e0c07946e15f\") " pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:39.688852 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.688732    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgk96\" (UniqueName: \"kubernetes.io/projected/1c1df2d1-4327-40cf-815c-e0c07946e15f-kube-api-access-sgk96\") pod \"maas-api-654967854b-8p7lc\" (UID: \"1c1df2d1-4327-40cf-815c-e0c07946e15f\") " pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:39.789196 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.789163    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c1df2d1-4327-40cf-815c-e0c07946e15f-openshift-service-ca\") pod \"maas-api-654967854b-8p7lc\" (UID: \"1c1df2d1-4327-40cf-815c-e0c07946e15f\") " pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:39.789391 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.789216    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgk96\" (UniqueName: \"kubernetes.io/projected/1c1df2d1-4327-40cf-815c-e0c07946e15f-kube-api-access-sgk96\") pod \"maas-api-654967854b-8p7lc\" (UID: \"1c1df2d1-4327-40cf-815c-e0c07946e15f\") " pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:39.789391 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.789301    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c1df2d1-4327-40cf-815c-e0c07946e15f-maas-api-tls\") pod \"maas-api-654967854b-8p7lc\" (UID: \"1c1df2d1-4327-40cf-815c-e0c07946e15f\") " pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:39.789774 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.789753    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c1df2d1-4327-40cf-815c-e0c07946e15f-openshift-service-ca\") pod \"maas-api-654967854b-8p7lc\" (UID: \"1c1df2d1-4327-40cf-815c-e0c07946e15f\") " pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:39.791680 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.791660    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c1df2d1-4327-40cf-815c-e0c07946e15f-maas-api-tls\") pod \"maas-api-654967854b-8p7lc\" (UID: \"1c1df2d1-4327-40cf-815c-e0c07946e15f\") " pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:39.800735 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.800714    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgk96\" (UniqueName: \"kubernetes.io/projected/1c1df2d1-4327-40cf-815c-e0c07946e15f-kube-api-access-sgk96\") pod \"maas-api-654967854b-8p7lc\" (UID: \"1c1df2d1-4327-40cf-815c-e0c07946e15f\") " pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:39.945569 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:39.945490    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:40.090128 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.090093    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-654967854b-8p7lc"]
Jul 10 02:20:40.092849 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:20:40.092822    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c1df2d1_4327_40cf_815c_e0c07946e15f.slice/crio-385770e2f9c503f79711eb45bf0673074f269420215df27782e5cb5d571f1382 WatchSource:0}: Error finding container 385770e2f9c503f79711eb45bf0673074f269420215df27782e5cb5d571f1382: Status 404 returned error can't find the container with id 385770e2f9c503f79711eb45bf0673074f269420215df27782e5cb5d571f1382
Jul 10 02:20:40.154348 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.154306    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-78tkn"]
Jul 10 02:20:40.159270 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.159248    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-78tkn"
Jul 10 02:20:40.160894 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.160873    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 10 02:20:40.167566 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.167542    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-z6jdx"]
Jul 10 02:20:40.170694 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.170673    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-78tkn"]
Jul 10 02:20:40.170793 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.170777    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-z6jdx"
Jul 10 02:20:40.173046 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.172991    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-j4nl7\""
Jul 10 02:20:40.181551 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.181529    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-z6jdx"]
Jul 10 02:20:40.191869 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.191841    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37f57f08-7bf6-4b56-abf5-a9526f547379-plugins-config-volume\") pod \"payload-pre-processing-7964457696-78tkn\" (UID: \"37f57f08-7bf6-4b56-abf5-a9526f547379\") " pod="openshift-ingress/payload-pre-processing-7964457696-78tkn"
Jul 10 02:20:40.191959 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.191893    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfg5l\" (UniqueName: \"kubernetes.io/projected/7d0aa707-2ba1-4753-9b07-c14793ab2861-kube-api-access-xfg5l\") pod \"payload-processing-58f86c7798-z6jdx\" (UID: \"7d0aa707-2ba1-4753-9b07-c14793ab2861\") " pod="openshift-ingress/payload-processing-58f86c7798-z6jdx"
Jul 10 02:20:40.191959 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.191931    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mxfb\" (UniqueName: \"kubernetes.io/projected/37f57f08-7bf6-4b56-abf5-a9526f547379-kube-api-access-5mxfb\") pod \"payload-pre-processing-7964457696-78tkn\" (UID: \"37f57f08-7bf6-4b56-abf5-a9526f547379\") " pod="openshift-ingress/payload-pre-processing-7964457696-78tkn"
Jul 10 02:20:40.192030 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.191990    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0aa707-2ba1-4753-9b07-c14793ab2861-plugins-config-volume\") pod \"payload-processing-58f86c7798-z6jdx\" (UID: \"7d0aa707-2ba1-4753-9b07-c14793ab2861\") " pod="openshift-ingress/payload-processing-58f86c7798-z6jdx"
Jul 10 02:20:40.219779 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.219702    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc" event={"ID":"1c1df2d1-4327-40cf-815c-e0c07946e15f","Type":"ContainerStarted","Data":"385770e2f9c503f79711eb45bf0673074f269420215df27782e5cb5d571f1382"}
Jul 10 02:20:40.292721 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.292677    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0aa707-2ba1-4753-9b07-c14793ab2861-plugins-config-volume\") pod \"payload-processing-58f86c7798-z6jdx\" (UID: \"7d0aa707-2ba1-4753-9b07-c14793ab2861\") " pod="openshift-ingress/payload-processing-58f86c7798-z6jdx"
Jul 10 02:20:40.292937 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.292747    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37f57f08-7bf6-4b56-abf5-a9526f547379-plugins-config-volume\") pod \"payload-pre-processing-7964457696-78tkn\" (UID: \"37f57f08-7bf6-4b56-abf5-a9526f547379\") " pod="openshift-ingress/payload-pre-processing-7964457696-78tkn"
Jul 10 02:20:40.292937 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.292797    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfg5l\" (UniqueName: \"kubernetes.io/projected/7d0aa707-2ba1-4753-9b07-c14793ab2861-kube-api-access-xfg5l\") pod \"payload-processing-58f86c7798-z6jdx\" (UID: \"7d0aa707-2ba1-4753-9b07-c14793ab2861\") " pod="openshift-ingress/payload-processing-58f86c7798-z6jdx"
Jul 10 02:20:40.293062 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.292928    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mxfb\" (UniqueName: \"kubernetes.io/projected/37f57f08-7bf6-4b56-abf5-a9526f547379-kube-api-access-5mxfb\") pod \"payload-pre-processing-7964457696-78tkn\" (UID: \"37f57f08-7bf6-4b56-abf5-a9526f547379\") " pod="openshift-ingress/payload-pre-processing-7964457696-78tkn"
Jul 10 02:20:40.293497 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.293479    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37f57f08-7bf6-4b56-abf5-a9526f547379-plugins-config-volume\") pod \"payload-pre-processing-7964457696-78tkn\" (UID: \"37f57f08-7bf6-4b56-abf5-a9526f547379\") " pod="openshift-ingress/payload-pre-processing-7964457696-78tkn"
Jul 10 02:20:40.293581 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.293516    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d0aa707-2ba1-4753-9b07-c14793ab2861-plugins-config-volume\") pod \"payload-processing-58f86c7798-z6jdx\" (UID: \"7d0aa707-2ba1-4753-9b07-c14793ab2861\") " pod="openshift-ingress/payload-processing-58f86c7798-z6jdx"
Jul 10 02:20:40.301763 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.301742    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mxfb\" (UniqueName: \"kubernetes.io/projected/37f57f08-7bf6-4b56-abf5-a9526f547379-kube-api-access-5mxfb\") pod \"payload-pre-processing-7964457696-78tkn\" (UID: \"37f57f08-7bf6-4b56-abf5-a9526f547379\") " pod="openshift-ingress/payload-pre-processing-7964457696-78tkn"
Jul 10 02:20:40.301885 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.301853    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfg5l\" (UniqueName: \"kubernetes.io/projected/7d0aa707-2ba1-4753-9b07-c14793ab2861-kube-api-access-xfg5l\") pod \"payload-processing-58f86c7798-z6jdx\" (UID: \"7d0aa707-2ba1-4753-9b07-c14793ab2861\") " pod="openshift-ingress/payload-processing-58f86c7798-z6jdx"
Jul 10 02:20:40.471886 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.471780    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-78tkn"
Jul 10 02:20:40.483013 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.482988    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-z6jdx"
Jul 10 02:20:40.635839 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.635785    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-78tkn"]
Jul 10 02:20:40.639918 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:20:40.639872    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37f57f08_7bf6_4b56_abf5_a9526f547379.slice/crio-e6d40ff9cbd12604f92def5bf0249d526ce17c477d108e6dc941a728d0b851ce WatchSource:0}: Error finding container e6d40ff9cbd12604f92def5bf0249d526ce17c477d108e6dc941a728d0b851ce: Status 404 returned error can't find the container with id e6d40ff9cbd12604f92def5bf0249d526ce17c477d108e6dc941a728d0b851ce
Jul 10 02:20:40.661152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:40.661132    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-z6jdx"]
Jul 10 02:20:40.663178 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:20:40.663155    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0aa707_2ba1_4753_9b07_c14793ab2861.slice/crio-5d01e67efbc10d5bde591b089391eea042b3072e60c099c8e107228fd11a33e8 WatchSource:0}: Error finding container 5d01e67efbc10d5bde591b089391eea042b3072e60c099c8e107228fd11a33e8: Status 404 returned error can't find the container with id 5d01e67efbc10d5bde591b089391eea042b3072e60c099c8e107228fd11a33e8
Jul 10 02:20:41.226550 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:41.226507    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-78tkn" event={"ID":"37f57f08-7bf6-4b56-abf5-a9526f547379","Type":"ContainerStarted","Data":"e6d40ff9cbd12604f92def5bf0249d526ce17c477d108e6dc941a728d0b851ce"}
Jul 10 02:20:41.227943 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:41.227915    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-z6jdx" event={"ID":"7d0aa707-2ba1-4753-9b07-c14793ab2861","Type":"ContainerStarted","Data":"5d01e67efbc10d5bde591b089391eea042b3072e60c099c8e107228fd11a33e8"}
Jul 10 02:20:43.238643 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:43.238597    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-78tkn" event={"ID":"37f57f08-7bf6-4b56-abf5-a9526f547379","Type":"ContainerStarted","Data":"1a08224ac97a96f775a53f872ff11b09d427e1f7b95f1be47878a190684d20cf"}
Jul 10 02:20:43.239153 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:43.238774    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-78tkn"
Jul 10 02:20:43.243715 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:43.243667    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-z6jdx" event={"ID":"7d0aa707-2ba1-4753-9b07-c14793ab2861","Type":"ContainerStarted","Data":"dae4e1659dc9705733f7dcce8710b714ba7476071f7c04b673435f6e12d22172"}
Jul 10 02:20:43.244435 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:43.244407    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-z6jdx"
Jul 10 02:20:43.245686 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:43.245658    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc" event={"ID":"1c1df2d1-4327-40cf-815c-e0c07946e15f","Type":"ContainerStarted","Data":"b1e4e88c0fb80e11e6da2368e3567a3ff2e72462905797239ab4934c4a177e98"}
Jul 10 02:20:43.246072 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:43.246046    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:43.256037 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:43.255994    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-78tkn" podStartSLOduration=1.432970777 podStartE2EDuration="3.255980277s" totalImagesPullingTime="1.8230095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:20:43.255180302 +0000 UTC m=+799.571490187" watchObservedRunningTime="2026-07-10 02:20:43.255980277 +0000 UTC m=+799.572290424"
Jul 10 02:20:43.273483 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:43.273433    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc" podStartSLOduration=1.905911561 podStartE2EDuration="4.273416927s" totalImagesPullingTime="2.367505366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:20:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:20:43.271967187 +0000 UTC m=+799.588277059" watchObservedRunningTime="2026-07-10 02:20:43.273416927 +0000 UTC m=+799.589726799"
Jul 10 02:20:43.286997 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:43.286946    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-z6jdx" podStartSLOduration=1.490269611 podStartE2EDuration="3.286927166s" totalImagesPullingTime="1.796657555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:20:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:20:43.286092909 +0000 UTC m=+799.602402782" watchObservedRunningTime="2026-07-10 02:20:43.286927166 +0000 UTC m=+799.603237039"
Jul 10 02:20:48.271449 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:48.271404    2557 generic.go:356] "Generic (PLEG): container finished" podID="964c11a5-ebcf-40b5-8ff5-9b8156cc6c24" containerID="cd006ade914df6cbd5144211761d05c487df6a9ddfdc631628d330a127cf7519" exitCode=0
Jul 10 02:20:48.271932 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:48.271479    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk" event={"ID":"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24","Type":"ContainerDied","Data":"cd006ade914df6cbd5144211761d05c487df6a9ddfdc631628d330a127cf7519"}
Jul 10 02:20:48.271932 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:48.271859    2557 scope.go:117] "RemoveContainer" containerID="cd006ade914df6cbd5144211761d05c487df6a9ddfdc631628d330a127cf7519"
Jul 10 02:20:48.272845 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:48.272793    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:20:48.272955 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:48.272886    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:20:48.272955 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:48.272920    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:20:48.810787 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:48.810753    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:20:49.254938 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:49.254871    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-654967854b-8p7lc"
Jul 10 02:20:49.279294 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:49.279255    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk" event={"ID":"964c11a5-ebcf-40b5-8ff5-9b8156cc6c24","Type":"ContainerStarted","Data":"29ca454e272c90e59b19213e59d5fde8d5d8eee0d1c007f5dfd4051bb00915d4"}
Jul 10 02:20:49.473213 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:49.473173    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:20:49.474394 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:49.474364    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xngqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Jul 10 02:20:49.474534 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:49.474433    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk" podUID="964c11a5-ebcf-40b5-8ff5-9b8156cc6c24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Jul 10 02:20:50.473825 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:50.473760    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xngqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Jul 10 02:20:50.474231 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:50.473846    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk" podUID="964c11a5-ebcf-40b5-8ff5-9b8156cc6c24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Jul 10 02:20:51.473488 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:51.473448    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xngqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Jul 10 02:20:51.473680 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:51.473507    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk" podUID="964c11a5-ebcf-40b5-8ff5-9b8156cc6c24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Jul 10 02:20:52.473825 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:52.473773    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xngqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Jul 10 02:20:52.474233 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:52.473857    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk" podUID="964c11a5-ebcf-40b5-8ff5-9b8156cc6c24" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Jul 10 02:20:53.477338 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:53.477305    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:20:53.477784 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:53.477594    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:20:53.478384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:53.478365    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xngqk"
Jul 10 02:20:54.251632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:54.251607    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-78tkn"
Jul 10 02:20:55.257176 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:20:55.257148    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-z6jdx"
Jul 10 02:21:02.355555 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.355514    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"]
Jul 10 02:21:02.359103 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.359079    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.361412 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.361387    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qlqbf\""
Jul 10 02:21:02.361412 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.361406    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 10 02:21:02.361593 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.361395    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 10 02:21:02.361777 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.361762    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 10 02:21:02.368934 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.368911    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"]
Jul 10 02:21:02.493707 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.493672    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.493707 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.493710    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.493976 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.493751    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.493976 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.493842    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlf6q\" (UniqueName: \"kubernetes.io/projected/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-kube-api-access-xlf6q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.493976 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.493871    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.494113 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.493996    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.595124 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.595085    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.595124 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.595127    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.595367 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.595147    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.595367 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.595170    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.595367 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.595208    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlf6q\" (UniqueName: \"kubernetes.io/projected/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-kube-api-access-xlf6q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.595367 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.595225    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.595581 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.595554    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.595635 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.595582    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.595686 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.595657    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.597539 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.597517    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.597776 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.597757    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.605451 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.605426    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlf6q\" (UniqueName: \"kubernetes.io/projected/269c6d58-4eaa-4f66-bdf1-fd697e7506b5-kube-api-access-xlf6q\") pod \"e2e-distinct-simulated-kserve-59975d69f9-r5sf6\" (UID: \"269c6d58-4eaa-4f66-bdf1-fd697e7506b5\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.670296 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.670219    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:02.811153 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:02.811127    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"]
Jul 10 02:21:02.813222 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:21:02.813174    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod269c6d58_4eaa_4f66_bdf1_fd697e7506b5.slice/crio-d6352c22c67ab276e3dc82066f649a4d6c08dcd8c2734d845bcc71579173ae5e WatchSource:0}: Error finding container d6352c22c67ab276e3dc82066f649a4d6c08dcd8c2734d845bcc71579173ae5e: Status 404 returned error can't find the container with id d6352c22c67ab276e3dc82066f649a4d6c08dcd8c2734d845bcc71579173ae5e
Jul 10 02:21:03.354608 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:03.354565    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6" event={"ID":"269c6d58-4eaa-4f66-bdf1-fd697e7506b5","Type":"ContainerStarted","Data":"d6352c22c67ab276e3dc82066f649a4d6c08dcd8c2734d845bcc71579173ae5e"}
Jul 10 02:21:04.360138 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:04.360100    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6" event={"ID":"269c6d58-4eaa-4f66-bdf1-fd697e7506b5","Type":"ContainerStarted","Data":"88cf358fd2209724e6fd4213b6bd4e0f910582d75d1d70fff9d21e84bced59b6"}
Jul 10 02:21:04.378175 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:04.378129    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6" podStartSLOduration=1.263803662 podStartE2EDuration="2.378114216s" totalImagesPullingTime="1.114310554s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:21:04.377058271 +0000 UTC m=+820.693368170" watchObservedRunningTime="2026-07-10 02:21:04.378114216 +0000 UTC m=+820.694424124"
Jul 10 02:21:12.670658 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:12.670619    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:12.671189 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:12.670678    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:12.683536 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:12.683511    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:13.415494 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:13.415459    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-r5sf6"
Jul 10 02:21:33.148959 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.148927    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"]
Jul 10 02:21:33.152582 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.152554    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.154597 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.154577    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 10 02:21:33.161441 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.161415    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"]
Jul 10 02:21:33.162183 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.162162    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.162251 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.162205    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.162309 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.162259    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.162431 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.162413    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.162482 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.162443    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.162482 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.162472    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsm9l\" (UniqueName: \"kubernetes.io/projected/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-kube-api-access-bsm9l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.263599 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.263566    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.263793 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.263605    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.263793 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.263670    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.263793 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.263696    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.263793 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.263721    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsm9l\" (UniqueName: \"kubernetes.io/projected/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-kube-api-access-bsm9l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.264022 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.263868    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.264222 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.264197    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.264222 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.264213    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.264351 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.264330    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.266418 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.266394    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.266596 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.266579    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.273681 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.273656    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsm9l\" (UniqueName: \"kubernetes.io/projected/aee94787-f990-4cc9-a3c0-e0efbe2e1ccc-kube-api-access-bsm9l\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr\" (UID: \"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.462991 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.462908    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:33.596089 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:33.596064    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"]
Jul 10 02:21:33.598167 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:21:33.598135    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaee94787_f990_4cc9_a3c0_e0efbe2e1ccc.slice/crio-7cdec63b757300dc18493235f22330ecb25385b08b51c47829ce2fe6ac974412 WatchSource:0}: Error finding container 7cdec63b757300dc18493235f22330ecb25385b08b51c47829ce2fe6ac974412: Status 404 returned error can't find the container with id 7cdec63b757300dc18493235f22330ecb25385b08b51c47829ce2fe6ac974412
Jul 10 02:21:34.504910 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:34.504869    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr" event={"ID":"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc","Type":"ContainerStarted","Data":"d7d6389da197188ba51f119b895054bfc75d0ecc500bc14816c0c1897aba7703"}
Jul 10 02:21:34.504910 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:34.504914    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr" event={"ID":"aee94787-f990-4cc9-a3c0-e0efbe2e1ccc","Type":"ContainerStarted","Data":"7cdec63b757300dc18493235f22330ecb25385b08b51c47829ce2fe6ac974412"}
Jul 10 02:21:34.523779 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:34.523728    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr" podStartSLOduration=1.3559168019999999 podStartE2EDuration="1.523713921s" totalImagesPullingTime="167.797119ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:21:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:21:34.521983215 +0000 UTC m=+850.838293099" watchObservedRunningTime="2026-07-10 02:21:34.523713921 +0000 UTC m=+850.840023793"
Jul 10 02:21:43.463714 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:43.463657    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:43.464215 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:43.463758    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:43.476246 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:43.476215    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:21:43.557365 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:21:43.557330    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr"
Jul 10 02:22:24.154737 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:24.154660    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:22:24.157394 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:24.157372    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:22:29.092406 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.092369    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b55d68598-pdhkg"]
Jul 10 02:22:29.095751 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.095726    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b55d68598-pdhkg"
Jul 10 02:22:29.113559 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.113534    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b55d68598-pdhkg"]
Jul 10 02:22:29.153828 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.153755    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppwr5\" (UniqueName: \"kubernetes.io/projected/751cb34d-d874-47ab-ae3c-883d57c5d863-kube-api-access-ppwr5\") pod \"authorino-7b55d68598-pdhkg\" (UID: \"751cb34d-d874-47ab-ae3c-883d57c5d863\") " pod="kuadrant-system/authorino-7b55d68598-pdhkg"
Jul 10 02:22:29.154004 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.153845    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/751cb34d-d874-47ab-ae3c-883d57c5d863-tls-cert\") pod \"authorino-7b55d68598-pdhkg\" (UID: \"751cb34d-d874-47ab-ae3c-883d57c5d863\") " pod="kuadrant-system/authorino-7b55d68598-pdhkg"
Jul 10 02:22:29.254429 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.254396    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppwr5\" (UniqueName: \"kubernetes.io/projected/751cb34d-d874-47ab-ae3c-883d57c5d863-kube-api-access-ppwr5\") pod \"authorino-7b55d68598-pdhkg\" (UID: \"751cb34d-d874-47ab-ae3c-883d57c5d863\") " pod="kuadrant-system/authorino-7b55d68598-pdhkg"
Jul 10 02:22:29.254591 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.254442    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/751cb34d-d874-47ab-ae3c-883d57c5d863-tls-cert\") pod \"authorino-7b55d68598-pdhkg\" (UID: \"751cb34d-d874-47ab-ae3c-883d57c5d863\") " pod="kuadrant-system/authorino-7b55d68598-pdhkg"
Jul 10 02:22:29.256787 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.256765    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/751cb34d-d874-47ab-ae3c-883d57c5d863-tls-cert\") pod \"authorino-7b55d68598-pdhkg\" (UID: \"751cb34d-d874-47ab-ae3c-883d57c5d863\") " pod="kuadrant-system/authorino-7b55d68598-pdhkg"
Jul 10 02:22:29.262999 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.262970    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppwr5\" (UniqueName: \"kubernetes.io/projected/751cb34d-d874-47ab-ae3c-883d57c5d863-kube-api-access-ppwr5\") pod \"authorino-7b55d68598-pdhkg\" (UID: \"751cb34d-d874-47ab-ae3c-883d57c5d863\") " pod="kuadrant-system/authorino-7b55d68598-pdhkg"
Jul 10 02:22:29.405297 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.405224    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b55d68598-pdhkg"
Jul 10 02:22:29.540142 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.540009    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b55d68598-pdhkg"]
Jul 10 02:22:29.542556 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:22:29.542532    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod751cb34d_d874_47ab_ae3c_883d57c5d863.slice/crio-631d36e3244288542db9c69346e1132f7a95f85462b41b1d94a72b90b06aa5f8 WatchSource:0}: Error finding container 631d36e3244288542db9c69346e1132f7a95f85462b41b1d94a72b90b06aa5f8: Status 404 returned error can't find the container with id 631d36e3244288542db9c69346e1132f7a95f85462b41b1d94a72b90b06aa5f8
Jul 10 02:22:29.766951 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:29.766831    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b55d68598-pdhkg" event={"ID":"751cb34d-d874-47ab-ae3c-883d57c5d863","Type":"ContainerStarted","Data":"631d36e3244288542db9c69346e1132f7a95f85462b41b1d94a72b90b06aa5f8"}
Jul 10 02:22:30.773869 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:30.773836    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b55d68598-pdhkg" event={"ID":"751cb34d-d874-47ab-ae3c-883d57c5d863","Type":"ContainerStarted","Data":"7276e22a420c39cecd8f6986bc6a959d0104f52c91c02f98ff0ff90e0a6ad73d"}
Jul 10 02:22:30.794603 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:30.794536    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b55d68598-pdhkg" podStartSLOduration=1.44736082 podStartE2EDuration="1.794499623s" totalImagesPullingTime="347.138803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:22:30.793751804 +0000 UTC m=+907.110061679" watchObservedRunningTime="2026-07-10 02:22:30.794499623 +0000 UTC m=+907.110809497"
Jul 10 02:22:30.827826 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:30.827441    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56b64b8547-zvwp7"]
Jul 10 02:22:30.827826 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:30.827677    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-56b64b8547-zvwp7" podUID="b85a3937-40ab-4db7-8ea1-fcd2303e5e72" containerName="authorino" containerID="cri-o://f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3" gracePeriod=30
Jul 10 02:22:31.107971 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.107945    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b64b8547-zvwp7"
Jul 10 02:22:31.174288 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.174252    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzl4v\" (UniqueName: \"kubernetes.io/projected/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-kube-api-access-bzl4v\") pod \"b85a3937-40ab-4db7-8ea1-fcd2303e5e72\" (UID: \"b85a3937-40ab-4db7-8ea1-fcd2303e5e72\") "
Jul 10 02:22:31.174453 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.174304    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-tls-cert\") pod \"b85a3937-40ab-4db7-8ea1-fcd2303e5e72\" (UID: \"b85a3937-40ab-4db7-8ea1-fcd2303e5e72\") "
Jul 10 02:22:31.176492 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.176460    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-kube-api-access-bzl4v" (OuterVolumeSpecName: "kube-api-access-bzl4v") pod "b85a3937-40ab-4db7-8ea1-fcd2303e5e72" (UID: "b85a3937-40ab-4db7-8ea1-fcd2303e5e72"). InnerVolumeSpecName "kube-api-access-bzl4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:22:31.184736 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.184702    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b85a3937-40ab-4db7-8ea1-fcd2303e5e72" (UID: "b85a3937-40ab-4db7-8ea1-fcd2303e5e72"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:22:31.275561 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.275529    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzl4v\" (UniqueName: \"kubernetes.io/projected/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-kube-api-access-bzl4v\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:22:31.275561 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.275558    2557 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b85a3937-40ab-4db7-8ea1-fcd2303e5e72-tls-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:22:31.780409 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.780375    2557 generic.go:356] "Generic (PLEG): container finished" podID="b85a3937-40ab-4db7-8ea1-fcd2303e5e72" containerID="f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3" exitCode=0
Jul 10 02:22:31.780867 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.780419    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56b64b8547-zvwp7"
Jul 10 02:22:31.780867 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.780422    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56b64b8547-zvwp7" event={"ID":"b85a3937-40ab-4db7-8ea1-fcd2303e5e72","Type":"ContainerDied","Data":"f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3"}
Jul 10 02:22:31.780867 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.780461    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56b64b8547-zvwp7" event={"ID":"b85a3937-40ab-4db7-8ea1-fcd2303e5e72","Type":"ContainerDied","Data":"e2222aeb28720afae7dc44f0373b4ccca6d427a03d0443cc72bf58fa2a4cb850"}
Jul 10 02:22:31.780867 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.780478    2557 scope.go:117] "RemoveContainer" containerID="f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3"
Jul 10 02:22:31.789715 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.789698    2557 scope.go:117] "RemoveContainer" containerID="f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3"
Jul 10 02:22:31.789990 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:22:31.789972    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3\": container with ID starting with f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3 not found: ID does not exist" containerID="f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3"
Jul 10 02:22:31.790044 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.790001    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3"} err="failed to get container status \"f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3\": rpc error: code = NotFound desc = could not find container \"f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3\": container with ID starting with f1999b9dd8d0c4ba7ccccc22344eff3fe5eae57177f0a27690a2a16484215ee3 not found: ID does not exist"
Jul 10 02:22:31.802940 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.802913    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56b64b8547-zvwp7"]
Jul 10 02:22:31.808859 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:31.808836    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56b64b8547-zvwp7"]
Jul 10 02:22:32.242612 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:22:32.242540    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b85a3937-40ab-4db7-8ea1-fcd2303e5e72" path="/var/lib/kubelet/pods/b85a3937-40ab-4db7-8ea1-fcd2303e5e72/volumes"
Jul 10 02:23:46.308574 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:46.308487    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56f677784f-65p67"]
Jul 10 02:23:46.309046 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:46.308751    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56f677784f-65p67" podUID="b308d089-1156-4914-a05a-477037a69c28" containerName="manager" containerID="cri-o://c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235" gracePeriod=30
Jul 10 02:23:46.556500 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:46.556476    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:23:46.605626 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:46.605544    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n66rg\" (UniqueName: \"kubernetes.io/projected/b308d089-1156-4914-a05a-477037a69c28-kube-api-access-n66rg\") pod \"b308d089-1156-4914-a05a-477037a69c28\" (UID: \"b308d089-1156-4914-a05a-477037a69c28\") "
Jul 10 02:23:46.605626 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:46.605618    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b308d089-1156-4914-a05a-477037a69c28-webhook-cert\") pod \"b308d089-1156-4914-a05a-477037a69c28\" (UID: \"b308d089-1156-4914-a05a-477037a69c28\") "
Jul 10 02:23:46.607615 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:46.607585    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b308d089-1156-4914-a05a-477037a69c28-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b308d089-1156-4914-a05a-477037a69c28" (UID: "b308d089-1156-4914-a05a-477037a69c28"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:23:46.607727 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:46.607684    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b308d089-1156-4914-a05a-477037a69c28-kube-api-access-n66rg" (OuterVolumeSpecName: "kube-api-access-n66rg") pod "b308d089-1156-4914-a05a-477037a69c28" (UID: "b308d089-1156-4914-a05a-477037a69c28"). InnerVolumeSpecName "kube-api-access-n66rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:23:46.707057 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:46.707024    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n66rg\" (UniqueName: \"kubernetes.io/projected/b308d089-1156-4914-a05a-477037a69c28-kube-api-access-n66rg\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:23:46.707057 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:46.707053    2557 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b308d089-1156-4914-a05a-477037a69c28-webhook-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:23:47.135122 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.135080    2557 generic.go:356] "Generic (PLEG): container finished" podID="b308d089-1156-4914-a05a-477037a69c28" containerID="c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235" exitCode=0
Jul 10 02:23:47.135324 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.135148    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56f677784f-65p67"
Jul 10 02:23:47.135324 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.135157    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56f677784f-65p67" event={"ID":"b308d089-1156-4914-a05a-477037a69c28","Type":"ContainerDied","Data":"c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235"}
Jul 10 02:23:47.135324 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.135232    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56f677784f-65p67" event={"ID":"b308d089-1156-4914-a05a-477037a69c28","Type":"ContainerDied","Data":"1b0e19aa45185a40a32c6039ca552a9356ff61e1d819b67fd4a1c4e735d3f352"}
Jul 10 02:23:47.135324 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.135255    2557 scope.go:117] "RemoveContainer" containerID="c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235"
Jul 10 02:23:47.146722 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.146538    2557 scope.go:117] "RemoveContainer" containerID="c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235"
Jul 10 02:23:47.147197 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:23:47.147173    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235\": container with ID starting with c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235 not found: ID does not exist" containerID="c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235"
Jul 10 02:23:47.147283 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.147209    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235"} err="failed to get container status \"c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235\": rpc error: code = NotFound desc = could not find container \"c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235\": container with ID starting with c5d16be46df15558b49715ae64796925930c0bd1aea2d9a87df4969483ea6235 not found: ID does not exist"
Jul 10 02:23:47.167610 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.167576    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56f677784f-65p67"]
Jul 10 02:23:47.172680 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.172658    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56f677784f-65p67"]
Jul 10 02:23:47.985307 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.985274    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56f677784f-5grmt"]
Jul 10 02:23:47.985799 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.985778    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b308d089-1156-4914-a05a-477037a69c28" containerName="manager"
Jul 10 02:23:47.985904 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.985817    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b308d089-1156-4914-a05a-477037a69c28" containerName="manager"
Jul 10 02:23:47.985904 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.985846    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b85a3937-40ab-4db7-8ea1-fcd2303e5e72" containerName="authorino"
Jul 10 02:23:47.985904 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.985854    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b85a3937-40ab-4db7-8ea1-fcd2303e5e72" containerName="authorino"
Jul 10 02:23:47.986047 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.985939    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b308d089-1156-4914-a05a-477037a69c28" containerName="manager"
Jul 10 02:23:47.986047 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.985954    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b85a3937-40ab-4db7-8ea1-fcd2303e5e72" containerName="authorino"
Jul 10 02:23:47.990165 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.990144    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56f677784f-5grmt"
Jul 10 02:23:47.992589 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.992554    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-9dx8t\""
Jul 10 02:23:47.992589 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.992571    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 10 02:23:47.992589 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.992557    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 10 02:23:47.996914 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:47.996890    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56f677784f-5grmt"]
Jul 10 02:23:48.120997 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:48.120960    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/05b08f09-b1d2-44b4-9ba2-691b72cbbc1a-webhook-cert\") pod \"maas-controller-56f677784f-5grmt\" (UID: \"05b08f09-b1d2-44b4-9ba2-691b72cbbc1a\") " pod="opendatahub/maas-controller-56f677784f-5grmt"
Jul 10 02:23:48.121175 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:48.121023    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns778\" (UniqueName: \"kubernetes.io/projected/05b08f09-b1d2-44b4-9ba2-691b72cbbc1a-kube-api-access-ns778\") pod \"maas-controller-56f677784f-5grmt\" (UID: \"05b08f09-b1d2-44b4-9ba2-691b72cbbc1a\") " pod="opendatahub/maas-controller-56f677784f-5grmt"
Jul 10 02:23:48.222169 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:48.222135    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/05b08f09-b1d2-44b4-9ba2-691b72cbbc1a-webhook-cert\") pod \"maas-controller-56f677784f-5grmt\" (UID: \"05b08f09-b1d2-44b4-9ba2-691b72cbbc1a\") " pod="opendatahub/maas-controller-56f677784f-5grmt"
Jul 10 02:23:48.222337 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:48.222195    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns778\" (UniqueName: \"kubernetes.io/projected/05b08f09-b1d2-44b4-9ba2-691b72cbbc1a-kube-api-access-ns778\") pod \"maas-controller-56f677784f-5grmt\" (UID: \"05b08f09-b1d2-44b4-9ba2-691b72cbbc1a\") " pod="opendatahub/maas-controller-56f677784f-5grmt"
Jul 10 02:23:48.224652 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:48.224630    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/05b08f09-b1d2-44b4-9ba2-691b72cbbc1a-webhook-cert\") pod \"maas-controller-56f677784f-5grmt\" (UID: \"05b08f09-b1d2-44b4-9ba2-691b72cbbc1a\") " pod="opendatahub/maas-controller-56f677784f-5grmt"
Jul 10 02:23:48.231610 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:48.231588    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns778\" (UniqueName: \"kubernetes.io/projected/05b08f09-b1d2-44b4-9ba2-691b72cbbc1a-kube-api-access-ns778\") pod \"maas-controller-56f677784f-5grmt\" (UID: \"05b08f09-b1d2-44b4-9ba2-691b72cbbc1a\") " pod="opendatahub/maas-controller-56f677784f-5grmt"
Jul 10 02:23:48.242364 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:48.242309    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b308d089-1156-4914-a05a-477037a69c28" path="/var/lib/kubelet/pods/b308d089-1156-4914-a05a-477037a69c28/volumes"
Jul 10 02:23:48.301747 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:48.301714    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56f677784f-5grmt"
Jul 10 02:23:48.444762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:48.444734    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56f677784f-5grmt"]
Jul 10 02:23:48.446385 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:23:48.446361    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05b08f09_b1d2_44b4_9ba2_691b72cbbc1a.slice/crio-29f4fd59131f7b8302963cca241dccc3d05b786cba3b610263dde55284e2643a WatchSource:0}: Error finding container 29f4fd59131f7b8302963cca241dccc3d05b786cba3b610263dde55284e2643a: Status 404 returned error can't find the container with id 29f4fd59131f7b8302963cca241dccc3d05b786cba3b610263dde55284e2643a
Jul 10 02:23:48.447671 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:48.447652    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:23:49.147441 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:49.147408    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56f677784f-5grmt" event={"ID":"05b08f09-b1d2-44b4-9ba2-691b72cbbc1a","Type":"ContainerStarted","Data":"d15d7fff61183409fa4fa5973152237bc23a2fe49ab69f093eb59b937d76a885"}
Jul 10 02:23:49.147864 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:49.147463    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56f677784f-5grmt" event={"ID":"05b08f09-b1d2-44b4-9ba2-691b72cbbc1a","Type":"ContainerStarted","Data":"29f4fd59131f7b8302963cca241dccc3d05b786cba3b610263dde55284e2643a"}
Jul 10 02:23:49.147864 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:49.147562    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56f677784f-5grmt"
Jul 10 02:23:49.165723 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:23:49.165678    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56f677784f-5grmt" podStartSLOduration=1.624088829 podStartE2EDuration="2.165663627s" totalImagesPullingTime="541.574798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:23:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:23:49.164468974 +0000 UTC m=+985.480778836" watchObservedRunningTime="2026-07-10 02:23:49.165663627 +0000 UTC m=+985.481973499"
Jul 10 02:24:00.154405 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:24:00.154369    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56f677784f-5grmt"
Jul 10 02:27:24.182601 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:27:24.182569    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:27:24.187007 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:27:24.186983    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:28:32.728655 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:32.728616    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"]
Jul 10 02:28:32.729240 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:32.728850    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w" podUID="0d4d7cdd-d147-45b4-8141-e9d79b5d98a6" containerName="manager" containerID="cri-o://8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc" gracePeriod=10
Jul 10 02:28:34.884885 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:34.884861    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:28:35.000727 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.000640    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gw87x\" (UniqueName: \"kubernetes.io/projected/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-kube-api-access-gw87x\") pod \"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6\" (UID: \"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6\") "
Jul 10 02:28:35.000917 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.000741    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-extensions-socket-volume\") pod \"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6\" (UID: \"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6\") "
Jul 10 02:28:35.001111 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.001089    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0d4d7cdd-d147-45b4-8141-e9d79b5d98a6" (UID: "0d4d7cdd-d147-45b4-8141-e9d79b5d98a6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:28:35.002717 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.002695    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-kube-api-access-gw87x" (OuterVolumeSpecName: "kube-api-access-gw87x") pod "0d4d7cdd-d147-45b4-8141-e9d79b5d98a6" (UID: "0d4d7cdd-d147-45b4-8141-e9d79b5d98a6"). InnerVolumeSpecName "kube-api-access-gw87x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:28:35.102367 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.102318    2557 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-extensions-socket-volume\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:28:35.102367 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.102359    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gw87x\" (UniqueName: \"kubernetes.io/projected/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6-kube-api-access-gw87x\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:28:35.535130 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.534626    2557 generic.go:356] "Generic (PLEG): container finished" podID="0d4d7cdd-d147-45b4-8141-e9d79b5d98a6" containerID="8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc" exitCode=0
Jul 10 02:28:35.535130 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.534852    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w" event={"ID":"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6","Type":"ContainerDied","Data":"8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc"}
Jul 10 02:28:35.535130 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.534958    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w" event={"ID":"0d4d7cdd-d147-45b4-8141-e9d79b5d98a6","Type":"ContainerDied","Data":"eadd63e5c003a61a5470408f6eae54d5fe3e8ecf909621da093fac2d5b18605a"}
Jul 10 02:28:35.535130 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.535024    2557 scope.go:117] "RemoveContainer" containerID="8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc"
Jul 10 02:28:35.535599 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.535577    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"
Jul 10 02:28:35.545561 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.545543    2557 scope.go:117] "RemoveContainer" containerID="8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc"
Jul 10 02:28:35.545846 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:28:35.545792    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc\": container with ID starting with 8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc not found: ID does not exist" containerID="8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc"
Jul 10 02:28:35.545903 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.545860    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc"} err="failed to get container status \"8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc\": rpc error: code = NotFound desc = could not find container \"8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc\": container with ID starting with 8e159d3b3e7db8e4e4cee5e4cc764a379779c0ceb1f0b893e150e14219c791fc not found: ID does not exist"
Jul 10 02:28:35.564423 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.564396    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"]
Jul 10 02:28:35.566474 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:35.566452    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-xl44w"]
Jul 10 02:28:36.241882 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:28:36.241852    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d4d7cdd-d147-45b4-8141-e9d79b5d98a6" path="/var/lib/kubelet/pods/0d4d7cdd-d147-45b4-8141-e9d79b5d98a6/volumes"
Jul 10 02:29:33.942546 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:33.942509    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"]
Jul 10 02:29:33.943027 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:33.942891    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d4d7cdd-d147-45b4-8141-e9d79b5d98a6" containerName="manager"
Jul 10 02:29:33.943027 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:33.942903    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d4d7cdd-d147-45b4-8141-e9d79b5d98a6" containerName="manager"
Jul 10 02:29:33.943027 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:33.942972    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d4d7cdd-d147-45b4-8141-e9d79b5d98a6" containerName="manager"
Jul 10 02:29:33.947081 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:33.947061    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"
Jul 10 02:29:33.948970 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:33.948949    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-w5jbk\""
Jul 10 02:29:33.957059 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:33.957039    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"]
Jul 10 02:29:34.105988 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.105945    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a5143d3-e139-4f28-8619-f9d5e2980a8f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t7rtp\" (UID: \"0a5143d3-e139-4f28-8619-f9d5e2980a8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"
Jul 10 02:29:34.106178 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.106006    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqd6d\" (UniqueName: \"kubernetes.io/projected/0a5143d3-e139-4f28-8619-f9d5e2980a8f-kube-api-access-rqd6d\") pod \"kuadrant-operator-controller-manager-86878c769f-t7rtp\" (UID: \"0a5143d3-e139-4f28-8619-f9d5e2980a8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"
Jul 10 02:29:34.206876 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.206763    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a5143d3-e139-4f28-8619-f9d5e2980a8f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t7rtp\" (UID: \"0a5143d3-e139-4f28-8619-f9d5e2980a8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"
Jul 10 02:29:34.206876 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.206848    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqd6d\" (UniqueName: \"kubernetes.io/projected/0a5143d3-e139-4f28-8619-f9d5e2980a8f-kube-api-access-rqd6d\") pod \"kuadrant-operator-controller-manager-86878c769f-t7rtp\" (UID: \"0a5143d3-e139-4f28-8619-f9d5e2980a8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"
Jul 10 02:29:34.207153 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.207132    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a5143d3-e139-4f28-8619-f9d5e2980a8f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-t7rtp\" (UID: \"0a5143d3-e139-4f28-8619-f9d5e2980a8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"
Jul 10 02:29:34.215526 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.215506    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqd6d\" (UniqueName: \"kubernetes.io/projected/0a5143d3-e139-4f28-8619-f9d5e2980a8f-kube-api-access-rqd6d\") pod \"kuadrant-operator-controller-manager-86878c769f-t7rtp\" (UID: \"0a5143d3-e139-4f28-8619-f9d5e2980a8f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"
Jul 10 02:29:34.257931 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.257907    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"
Jul 10 02:29:34.394154 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.394129    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"]
Jul 10 02:29:34.395944 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:29:34.395920    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a5143d3_e139_4f28_8619_f9d5e2980a8f.slice/crio-6fd1549a1259139fcd50b83b93036e274a74dabd3a63b43662e3cee153bc7809 WatchSource:0}: Error finding container 6fd1549a1259139fcd50b83b93036e274a74dabd3a63b43662e3cee153bc7809: Status 404 returned error can't find the container with id 6fd1549a1259139fcd50b83b93036e274a74dabd3a63b43662e3cee153bc7809
Jul 10 02:29:34.809489 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.809454    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp" event={"ID":"0a5143d3-e139-4f28-8619-f9d5e2980a8f","Type":"ContainerStarted","Data":"1bea3f7fe307c81b9ac84b28535919e22b3d79852eb44f8cef43e6b2b9bf49fa"}
Jul 10 02:29:34.809653 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.809510    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp" event={"ID":"0a5143d3-e139-4f28-8619-f9d5e2980a8f","Type":"ContainerStarted","Data":"6fd1549a1259139fcd50b83b93036e274a74dabd3a63b43662e3cee153bc7809"}
Jul 10 02:29:34.809653 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.809585    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"
Jul 10 02:29:34.829712 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:34.829666    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp" podStartSLOduration=1.829652468 podStartE2EDuration="1.829652468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:29:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:29:34.82825506 +0000 UTC m=+1331.144564945" watchObservedRunningTime="2026-07-10 02:29:34.829652468 +0000 UTC m=+1331.145962340"
Jul 10 02:29:45.815861 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:29:45.815827    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-t7rtp"
Jul 10 02:32:24.209653 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:32:24.209609    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:32:24.216248 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:32:24.216225    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:36:05.156377 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.156280    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"]
Jul 10 02:36:05.160396 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.160375    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.163143 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.163120    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0a1b34cb-openshift-default-dockercfg-8gqjp\""
Jul 10 02:36:05.178797 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.178772    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"]
Jul 10 02:36:05.229627 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.229584    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-certs\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.229627 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.229630    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-istio-token\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.229877 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.229657    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a516b03d-0e08-4039-b156-858c007cd495-istiod-ca-cert\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.229877 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.229716    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-credential-socket\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.229877 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.229756    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a516b03d-0e08-4039-b156-858c007cd495-istio-podinfo\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.229877 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.229788    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-data\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.229877 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.229849    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-socket\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.229877 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.229866    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v97p\" (UniqueName: \"kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-kube-api-access-8v97p\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.230064 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.229929    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-envoy\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331054 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331014    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-socket\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331227 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331058    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v97p\" (UniqueName: \"kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-kube-api-access-8v97p\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331227 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331183    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-envoy\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331348 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331268    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-certs\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331348 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331297    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-istio-token\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331348 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331320    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a516b03d-0e08-4039-b156-858c007cd495-istiod-ca-cert\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331509 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331356    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-credential-socket\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331509 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331377    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a516b03d-0e08-4039-b156-858c007cd495-istio-podinfo\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331509 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331402    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-socket\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331509 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331414    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-data\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331693 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331635    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-certs\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331861 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331839    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-data\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.331926 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.331881    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-credential-socket\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.332040 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.332014    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a516b03d-0e08-4039-b156-858c007cd495-istiod-ca-cert\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.333446 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.333424    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-envoy\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.333702 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.333686    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a516b03d-0e08-4039-b156-858c007cd495-istio-podinfo\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.339946 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.339926    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-istio-token\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.340038 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.339948    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v97p\" (UniqueName: \"kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-kube-api-access-8v97p\") pod \"e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") " pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.471233 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.471150    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:05.606434 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.606394    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"]
Jul 10 02:36:05.608775 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:36:05.608728    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda516b03d_0e08_4039_b156_858c007cd495.slice/crio-4b4c1b604ccdb2d4f6207ae7a6077667567b2da07a85c203cd49a8d0aa5ba2c5 WatchSource:0}: Error finding container 4b4c1b604ccdb2d4f6207ae7a6077667567b2da07a85c203cd49a8d0aa5ba2c5: Status 404 returned error can't find the container with id 4b4c1b604ccdb2d4f6207ae7a6077667567b2da07a85c203cd49a8d0aa5ba2c5
Jul 10 02:36:05.611119 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.611088    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:36:05.611223 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.611167    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:36:05.611223 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.611215    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:36:05.714464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.714427    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6" event={"ID":"a516b03d-0e08-4039-b156-858c007cd495","Type":"ContainerStarted","Data":"4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024"}
Jul 10 02:36:05.714782 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.714507    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6" event={"ID":"a516b03d-0e08-4039-b156-858c007cd495","Type":"ContainerStarted","Data":"4b4c1b604ccdb2d4f6207ae7a6077667567b2da07a85c203cd49a8d0aa5ba2c5"}
Jul 10 02:36:05.734021 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:05.733925    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6" podStartSLOduration=0.733908677 podStartE2EDuration="733.908677ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:36:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:36:05.732476138 +0000 UTC m=+1722.048786010" watchObservedRunningTime="2026-07-10 02:36:05.733908677 +0000 UTC m=+1722.050218549"
Jul 10 02:36:06.471880 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.471847    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:06.476614 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.476588    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:06.484912 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.484883    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"]
Jul 10 02:36:06.488424 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.488406    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:06.490312 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.490296    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0a1b34cb-serving-cert\""
Jul 10 02:36:06.499343 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.499314    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"]
Jul 10 02:36:06.546906 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.546872    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-maas-api-tls\") pod \"maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:06.547071 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.546911    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h25w6\" (UniqueName: \"kubernetes.io/projected/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-kube-api-access-h25w6\") pod \"maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:06.547114 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.547088    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-openshift-service-ca\") pod \"maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:06.647885 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.647853    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-openshift-service-ca\") pod \"maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:06.648077 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.647909    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-maas-api-tls\") pod \"maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:06.648077 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.647930    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h25w6\" (UniqueName: \"kubernetes.io/projected/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-kube-api-access-h25w6\") pod \"maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:06.648202 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:36:06.648078    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-serving-cert: secret "maas-api-e2e-ait-0a1b34cb-serving-cert" not found
Jul 10 02:36:06.648202 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:36:06.648158    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-maas-api-tls podName:3e133b3e-29a0-414a-ab3e-a99ec68ac88d nodeName:}" failed. No retries permitted until 2026-07-10 02:36:07.148137093 +0000 UTC m=+1723.464446956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-maas-api-tls") pod "maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v" (UID: "3e133b3e-29a0-414a-ab3e-a99ec68ac88d") : secret "maas-api-e2e-ait-0a1b34cb-serving-cert" not found
Jul 10 02:36:06.648489 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.648468    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-openshift-service-ca\") pod \"maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:06.660590 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.660568    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h25w6\" (UniqueName: \"kubernetes.io/projected/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-kube-api-access-h25w6\") pod \"maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:06.719197 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.719168    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:06.720047 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:06.720030    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:07.153257 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:07.153165    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-maas-api-tls\") pod \"maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:07.155570 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:07.155542    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-maas-api-tls\") pod \"maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:07.402031 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:07.401990    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:07.550246 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:07.550041    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"]
Jul 10 02:36:07.552986 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:36:07.552960    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e133b3e_29a0_414a_ab3e_a99ec68ac88d.slice/crio-bff39f888b6b59e93e5c564b57910fde850461a4a7feb1d5d6e2b1b63ca98790 WatchSource:0}: Error finding container bff39f888b6b59e93e5c564b57910fde850461a4a7feb1d5d6e2b1b63ca98790: Status 404 returned error can't find the container with id bff39f888b6b59e93e5c564b57910fde850461a4a7feb1d5d6e2b1b63ca98790
Jul 10 02:36:07.554434 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:07.554416    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:36:07.725833 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:07.725723    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v" event={"ID":"3e133b3e-29a0-414a-ab3e-a99ec68ac88d","Type":"ContainerStarted","Data":"bff39f888b6b59e93e5c564b57910fde850461a4a7feb1d5d6e2b1b63ca98790"}
Jul 10 02:36:08.731738 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:08.731707    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v" event={"ID":"3e133b3e-29a0-414a-ab3e-a99ec68ac88d","Type":"ContainerStarted","Data":"96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6"}
Jul 10 02:36:08.750647 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:08.750596    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v" podStartSLOduration=2.175855784 podStartE2EDuration="2.750580144s" totalImagesPullingTime="574.72436ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:36:08.748994918 +0000 UTC m=+1725.065304790" watchObservedRunningTime="2026-07-10 02:36:08.750580144 +0000 UTC m=+1725.066890015"
Jul 10 02:36:09.741605 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:09.741572    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:15.751888 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:15.751859    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:36:18.787296 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:18.787263    2557 generic.go:356] "Generic (PLEG): container finished" podID="a516b03d-0e08-4039-b156-858c007cd495" containerID="4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024" exitCode=0
Jul 10 02:36:18.787783 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:18.787338    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6" event={"ID":"a516b03d-0e08-4039-b156-858c007cd495","Type":"ContainerDied","Data":"4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024"}
Jul 10 02:36:18.787783 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:18.787729    2557 scope.go:117] "RemoveContainer" containerID="4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024"
Jul 10 02:36:18.788722 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:18.788695    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:36:18.788854 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:18.788761    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:36:18.788854 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:18.788792    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:36:19.795080 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:19.795034    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6" event={"ID":"a516b03d-0e08-4039-b156-858c007cd495","Type":"ContainerStarted","Data":"836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88"}
Jul 10 02:36:20.472124 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:20.472090    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:20.473318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:20.473290    2557 patch_prober.go:28] interesting pod/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 10 02:36:20.473472 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:20.473341    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6" podUID="a516b03d-0e08-4039-b156-858c007cd495" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 10 02:36:21.472479 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:21.472446    2557 patch_prober.go:28] interesting pod/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 10 02:36:21.472864 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:21.472503    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6" podUID="a516b03d-0e08-4039-b156-858c007cd495" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 10 02:36:21.720000 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:21.719958    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:22.472145 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:22.472105    2557 patch_prober.go:28] interesting pod/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 10 02:36:22.472334 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:22.472175    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6" podUID="a516b03d-0e08-4039-b156-858c007cd495" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 10 02:36:23.475654 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:23.475625    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:36:23.476575 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:36:23.476558    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:37:11.555400 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:11.555367    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"]
Jul 10 02:37:11.555972 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:11.555584    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6" podUID="a516b03d-0e08-4039-b156-858c007cd495" containerName="istio-proxy" containerID="cri-o://836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88" gracePeriod=30
Jul 10 02:37:12.038861 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.038796    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"]
Jul 10 02:37:12.039111 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.039085    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v" podUID="3e133b3e-29a0-414a-ab3e-a99ec68ac88d" containerName="maas-api" containerID="cri-o://96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6" gracePeriod=30
Jul 10 02:37:12.293433 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.293364    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:37:12.464524 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.464492    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-maas-api-tls\") pod \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") "
Jul 10 02:37:12.464524 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.464531    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-openshift-service-ca\") pod \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") "
Jul 10 02:37:12.464778 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.464555    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h25w6\" (UniqueName: \"kubernetes.io/projected/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-kube-api-access-h25w6\") pod \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\" (UID: \"3e133b3e-29a0-414a-ab3e-a99ec68ac88d\") "
Jul 10 02:37:12.464934 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.464911    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3e133b3e-29a0-414a-ab3e-a99ec68ac88d" (UID: "3e133b3e-29a0-414a-ab3e-a99ec68ac88d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:37:12.466784 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.466762    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3e133b3e-29a0-414a-ab3e-a99ec68ac88d" (UID: "3e133b3e-29a0-414a-ab3e-a99ec68ac88d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:37:12.466956 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.466935    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-kube-api-access-h25w6" (OuterVolumeSpecName: "kube-api-access-h25w6") pod "3e133b3e-29a0-414a-ab3e-a99ec68ac88d" (UID: "3e133b3e-29a0-414a-ab3e-a99ec68ac88d"). InnerVolumeSpecName "kube-api-access-h25w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:37:12.565868 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.565755    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-maas-api-tls\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:12.565868 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.565789    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-openshift-service-ca\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:12.565868 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:12.565830    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h25w6\" (UniqueName: \"kubernetes.io/projected/3e133b3e-29a0-414a-ab3e-a99ec68ac88d-kube-api-access-h25w6\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:13.052796 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:13.052757    2557 generic.go:356] "Generic (PLEG): container finished" podID="3e133b3e-29a0-414a-ab3e-a99ec68ac88d" containerID="96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6" exitCode=0
Jul 10 02:37:13.052796 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:13.052816    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"
Jul 10 02:37:13.053072 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:13.052845    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v" event={"ID":"3e133b3e-29a0-414a-ab3e-a99ec68ac88d","Type":"ContainerDied","Data":"96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6"}
Jul 10 02:37:13.053072 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:13.052904    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v" event={"ID":"3e133b3e-29a0-414a-ab3e-a99ec68ac88d","Type":"ContainerDied","Data":"bff39f888b6b59e93e5c564b57910fde850461a4a7feb1d5d6e2b1b63ca98790"}
Jul 10 02:37:13.053072 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:13.052930    2557 scope.go:117] "RemoveContainer" containerID="96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6"
Jul 10 02:37:13.061471 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:13.061453    2557 scope.go:117] "RemoveContainer" containerID="96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6"
Jul 10 02:37:13.061745 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:37:13.061723    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6\": container with ID starting with 96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6 not found: ID does not exist" containerID="96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6"
Jul 10 02:37:13.061870 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:13.061751    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6"} err="failed to get container status \"96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6\": rpc error: code = NotFound desc = could not find container \"96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6\": container with ID starting with 96ab533aee08931bd3150ac5b4e6ef8962612882e71ca68bb3d1b965249f0bc6 not found: ID does not exist"
Jul 10 02:37:13.075916 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:13.075889    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"]
Jul 10 02:37:13.086828 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:13.082002    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0a1b34cb-6755b46cfc-7p84v"]
Jul 10 02:37:14.242875 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:14.242839    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e133b3e-29a0-414a-ab3e-a99ec68ac88d" path="/var/lib/kubelet/pods/3e133b3e-29a0-414a-ab3e-a99ec68ac88d/volumes"
Jul 10 02:37:16.820150 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:16.820090    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:37:17.008520 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.008488    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a516b03d-0e08-4039-b156-858c007cd495-istiod-ca-cert\") pod \"a516b03d-0e08-4039-b156-858c007cd495\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") "
Jul 10 02:37:17.008697 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.008534    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-socket\") pod \"a516b03d-0e08-4039-b156-858c007cd495\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") "
Jul 10 02:37:17.008697 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.008565    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a516b03d-0e08-4039-b156-858c007cd495-istio-podinfo\") pod \"a516b03d-0e08-4039-b156-858c007cd495\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") "
Jul 10 02:37:17.008697 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.008584    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-data\") pod \"a516b03d-0e08-4039-b156-858c007cd495\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") "
Jul 10 02:37:17.008915 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.008770    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-envoy\") pod \"a516b03d-0e08-4039-b156-858c007cd495\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") "
Jul 10 02:37:17.008915 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.008878    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v97p\" (UniqueName: \"kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-kube-api-access-8v97p\") pod \"a516b03d-0e08-4039-b156-858c007cd495\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") "
Jul 10 02:37:17.008915 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.008910    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-istio-token\") pod \"a516b03d-0e08-4039-b156-858c007cd495\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") "
Jul 10 02:37:17.009066 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.008881    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a516b03d-0e08-4039-b156-858c007cd495" (UID: "a516b03d-0e08-4039-b156-858c007cd495"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:37:17.009066 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.008984    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-certs\") pod \"a516b03d-0e08-4039-b156-858c007cd495\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") "
Jul 10 02:37:17.009066 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.009032    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a516b03d-0e08-4039-b156-858c007cd495-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a516b03d-0e08-4039-b156-858c007cd495" (UID: "a516b03d-0e08-4039-b156-858c007cd495"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:37:17.009066 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.009036    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-credential-socket\") pod \"a516b03d-0e08-4039-b156-858c007cd495\" (UID: \"a516b03d-0e08-4039-b156-858c007cd495\") "
Jul 10 02:37:17.009335 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.009311    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a516b03d-0e08-4039-b156-858c007cd495" (UID: "a516b03d-0e08-4039-b156-858c007cd495"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:37:17.009417 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.009346    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a516b03d-0e08-4039-b156-858c007cd495" (UID: "a516b03d-0e08-4039-b156-858c007cd495"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:37:17.009532 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.009514    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:17.009578 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.009537    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-credential-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:17.009578 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.009552    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a516b03d-0e08-4039-b156-858c007cd495-istiod-ca-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:17.009578 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.009565    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-workload-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:17.009681 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.009615    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-data" (OuterVolumeSpecName: "istio-data") pod "a516b03d-0e08-4039-b156-858c007cd495" (UID: "a516b03d-0e08-4039-b156-858c007cd495"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:37:17.011153 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.011107    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-kube-api-access-8v97p" (OuterVolumeSpecName: "kube-api-access-8v97p") pod "a516b03d-0e08-4039-b156-858c007cd495" (UID: "a516b03d-0e08-4039-b156-858c007cd495"). InnerVolumeSpecName "kube-api-access-8v97p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:37:17.011279 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.011201    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-istio-token" (OuterVolumeSpecName: "istio-token") pod "a516b03d-0e08-4039-b156-858c007cd495" (UID: "a516b03d-0e08-4039-b156-858c007cd495"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:37:17.011417 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.011396    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a516b03d-0e08-4039-b156-858c007cd495" (UID: "a516b03d-0e08-4039-b156-858c007cd495"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:37:17.011480 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.011448    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a516b03d-0e08-4039-b156-858c007cd495-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a516b03d-0e08-4039-b156-858c007cd495" (UID: "a516b03d-0e08-4039-b156-858c007cd495"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:37:17.074384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.074294    2557 generic.go:356] "Generic (PLEG): container finished" podID="a516b03d-0e08-4039-b156-858c007cd495" containerID="836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88" exitCode=0
Jul 10 02:37:17.074384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.074345    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"
Jul 10 02:37:17.074577 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.074378    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6" event={"ID":"a516b03d-0e08-4039-b156-858c007cd495","Type":"ContainerDied","Data":"836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88"}
Jul 10 02:37:17.074577 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.074435    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6" event={"ID":"a516b03d-0e08-4039-b156-858c007cd495","Type":"ContainerDied","Data":"4b4c1b604ccdb2d4f6207ae7a6077667567b2da07a85c203cd49a8d0aa5ba2c5"}
Jul 10 02:37:17.074577 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.074461    2557 scope.go:117] "RemoveContainer" containerID="836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88"
Jul 10 02:37:17.083330 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.083313    2557 scope.go:117] "RemoveContainer" containerID="4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024"
Jul 10 02:37:17.091313 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.091290    2557 scope.go:117] "RemoveContainer" containerID="836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88"
Jul 10 02:37:17.091579 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:37:17.091558    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88\": container with ID starting with 836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88 not found: ID does not exist" containerID="836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88"
Jul 10 02:37:17.091646 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.091590    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88"} err="failed to get container status \"836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88\": rpc error: code = NotFound desc = could not find container \"836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88\": container with ID starting with 836e1f9a215f22ed9a931e67886eb27b953a9d4adf9fa49d9cbb3020d3ad4e88 not found: ID does not exist"
Jul 10 02:37:17.091646 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.091609    2557 scope.go:117] "RemoveContainer" containerID="4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024"
Jul 10 02:37:17.091859 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:37:17.091845    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024\": container with ID starting with 4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024 not found: ID does not exist" containerID="4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024"
Jul 10 02:37:17.091914 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.091864    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024"} err="failed to get container status \"4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024\": rpc error: code = NotFound desc = could not find container \"4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024\": container with ID starting with 4d891d6ee97b7a5fa0b37918099bd40a09e1e93a35b334a41ef3e6ac5a8fa024 not found: ID does not exist"
Jul 10 02:37:17.097270 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.097248    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"]
Jul 10 02:37:17.100394 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.100374    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0a1b34cb-openshift-default-594cd6f8c5-2mtf6"]
Jul 10 02:37:17.110438 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.110421    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a516b03d-0e08-4039-b156-858c007cd495-istio-podinfo\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:17.110438 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.110439    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-data\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:17.110555 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.110451    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a516b03d-0e08-4039-b156-858c007cd495-istio-envoy\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:17.110555 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.110462    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v97p\" (UniqueName: \"kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-kube-api-access-8v97p\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:17.110555 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:17.110470    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a516b03d-0e08-4039-b156-858c007cd495-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:37:18.242634 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.242604    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a516b03d-0e08-4039-b156-858c007cd495" path="/var/lib/kubelet/pods/a516b03d-0e08-4039-b156-858c007cd495/volumes"
Jul 10 02:37:18.434746 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.434663    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"]
Jul 10 02:37:18.435143 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.435125    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e133b3e-29a0-414a-ab3e-a99ec68ac88d" containerName="maas-api"
Jul 10 02:37:18.435234 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.435146    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e133b3e-29a0-414a-ab3e-a99ec68ac88d" containerName="maas-api"
Jul 10 02:37:18.435234 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.435163    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a516b03d-0e08-4039-b156-858c007cd495" containerName="istio-proxy"
Jul 10 02:37:18.435234 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.435171    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a516b03d-0e08-4039-b156-858c007cd495" containerName="istio-proxy"
Jul 10 02:37:18.435234 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.435185    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a516b03d-0e08-4039-b156-858c007cd495" containerName="istio-proxy"
Jul 10 02:37:18.435234 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.435193    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a516b03d-0e08-4039-b156-858c007cd495" containerName="istio-proxy"
Jul 10 02:37:18.435470 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.435278    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e133b3e-29a0-414a-ab3e-a99ec68ac88d" containerName="maas-api"
Jul 10 02:37:18.435470 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.435294    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a516b03d-0e08-4039-b156-858c007cd495" containerName="istio-proxy"
Jul 10 02:37:18.440220 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.440197    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.442125 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.442108    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bc69ef18-openshift-default-dockercfg-6lvp6\""
Jul 10 02:37:18.450139 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.450118    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"]
Jul 10 02:37:18.523823 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.523780    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-envoy\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.524005 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.523836    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-podinfo\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.524005 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.523899    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istiod-ca-cert\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.524005 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.523938    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-certs\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.524005 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.523963    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-socket\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.524005 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.523981    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-data\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.524005 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.524001    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-credential-socket\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.524222 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.524062    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghqrm\" (UniqueName: \"kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-kube-api-access-ghqrm\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.524222 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.524102    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-token\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.624520 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.624482    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istiod-ca-cert\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.624520 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.624535    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-certs\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.624798 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.624578    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-socket\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.624798 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.624676    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-data\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.624798 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.624731    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-credential-socket\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.624798 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.624771    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghqrm\" (UniqueName: \"kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-kube-api-access-ghqrm\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.624798 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.624830    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-token\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.625140 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.624904    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-envoy\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.625140 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.624939    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-podinfo\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.625140 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.625009    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-socket\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.625140 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.625050    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-certs\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.625332 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.625291    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istiod-ca-cert\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.625453 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.625431    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-data\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.625615 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.625593    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-credential-socket\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.627366 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.627340    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-envoy\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.627453 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.627429    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-podinfo\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.633782 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.633759    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-token\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.633886 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.633873    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghqrm\" (UniqueName: \"kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-kube-api-access-ghqrm\") pod \"e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") " pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.751870 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.751771    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:18.891724 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.891690    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"]
Jul 10 02:37:18.894954 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:37:18.894914    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395a5a58_da99_4d1d_8a5c_f8028e7d8247.slice/crio-4ea5ccc792375a5f18997080eb7247281b82de706ea0543333f95d7b0f05ff41 WatchSource:0}: Error finding container 4ea5ccc792375a5f18997080eb7247281b82de706ea0543333f95d7b0f05ff41: Status 404 returned error can't find the container with id 4ea5ccc792375a5f18997080eb7247281b82de706ea0543333f95d7b0f05ff41
Jul 10 02:37:18.897318 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.897288    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:37:18.897382 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.897355    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:37:18.897425 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:18.897385    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:37:19.088029 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.087994    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" event={"ID":"395a5a58-da99-4d1d-8a5c-f8028e7d8247","Type":"ContainerStarted","Data":"79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811"}
Jul 10 02:37:19.088233 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.088059    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" event={"ID":"395a5a58-da99-4d1d-8a5c-f8028e7d8247","Type":"ContainerStarted","Data":"4ea5ccc792375a5f18997080eb7247281b82de706ea0543333f95d7b0f05ff41"}
Jul 10 02:37:19.664933 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.664874    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" podStartSLOduration=1.664849266 podStartE2EDuration="1.664849266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:37:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:37:19.123599844 +0000 UTC m=+1795.439909716" watchObservedRunningTime="2026-07-10 02:37:19.664849266 +0000 UTC m=+1795.981159140"
Jul 10 02:37:19.666734 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.666707    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"]
Jul 10 02:37:19.667416 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.667396    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a516b03d-0e08-4039-b156-858c007cd495" containerName="istio-proxy"
Jul 10 02:37:19.671006 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.670987    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:19.673109 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.673091    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-bc69ef18-serving-cert\""
Jul 10 02:37:19.679717 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.679692    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"]
Jul 10 02:37:19.732119 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.732087    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-maas-api-tls\") pod \"maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:19.732329 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.732130    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-openshift-service-ca\") pod \"maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:19.732329 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.732150    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptckg\" (UniqueName: \"kubernetes.io/projected/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-kube-api-access-ptckg\") pod \"maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:19.752892 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.752865    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:19.757524 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.757503    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:19.833144 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.833111    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-maas-api-tls\") pod \"maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:19.833295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.833156    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-openshift-service-ca\") pod \"maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:19.833295 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:37:19.833245    2557 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-serving-cert: secret "maas-api-e2e-ait-bc69ef18-serving-cert" not found
Jul 10 02:37:19.833295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.833276    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptckg\" (UniqueName: \"kubernetes.io/projected/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-kube-api-access-ptckg\") pod \"maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:19.833450 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:37:19.833305    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-maas-api-tls podName:c5dd28d6-41f5-4ee5-ac8e-425b548d2470 nodeName:}" failed. No retries permitted until 2026-07-10 02:37:20.333288988 +0000 UTC m=+1796.649598838 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-maas-api-tls") pod "maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9" (UID: "c5dd28d6-41f5-4ee5-ac8e-425b548d2470") : secret "maas-api-e2e-ait-bc69ef18-serving-cert" not found
Jul 10 02:37:19.833709 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.833692    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-openshift-service-ca\") pod \"maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:19.847596 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:19.847565    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptckg\" (UniqueName: \"kubernetes.io/projected/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-kube-api-access-ptckg\") pod \"maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:20.093411 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:20.093375    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:20.094441 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:20.094420    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:20.336737 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:20.336701    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-maas-api-tls\") pod \"maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:20.339143 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:20.339113    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-maas-api-tls\") pod \"maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:20.582886 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:20.582846    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:20.721613 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:20.721587    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"]
Jul 10 02:37:20.724017 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:37:20.723975    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5dd28d6_41f5_4ee5_ac8e_425b548d2470.slice/crio-fab5e120f4427c451946ce37c34dbeb3731551a341e5b14178e83c2ad0d884df WatchSource:0}: Error finding container fab5e120f4427c451946ce37c34dbeb3731551a341e5b14178e83c2ad0d884df: Status 404 returned error can't find the container with id fab5e120f4427c451946ce37c34dbeb3731551a341e5b14178e83c2ad0d884df
Jul 10 02:37:21.101346 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:21.101311    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9" event={"ID":"c5dd28d6-41f5-4ee5-ac8e-425b548d2470","Type":"ContainerStarted","Data":"fab5e120f4427c451946ce37c34dbeb3731551a341e5b14178e83c2ad0d884df"}
Jul 10 02:37:22.108315 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:22.108283    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9" event={"ID":"c5dd28d6-41f5-4ee5-ac8e-425b548d2470","Type":"ContainerStarted","Data":"4e168ecfe9785355c6afe851b57645aa61ec04c2da5bc418eed2c166b446222b"}
Jul 10 02:37:22.108792 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:22.108535    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:22.124820 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:22.124756    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9" podStartSLOduration=2.787141102 podStartE2EDuration="3.12473835s" totalImagesPullingTime="337.597248ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:37:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:37:22.123751202 +0000 UTC m=+1798.440061080" watchObservedRunningTime="2026-07-10 02:37:22.12473835 +0000 UTC m=+1798.441048223"
Jul 10 02:37:24.238965 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:24.238935    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:37:24.250033 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:24.250004    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:37:28.122151 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:28.122119    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:37:35.094770 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:35.094729    2557 patch_prober.go:28] interesting pod/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 10 02:37:35.095281 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:35.094786    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 10 02:37:35.173721 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:35.173682    2557 generic.go:356] "Generic (PLEG): container finished" podID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerID="79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811" exitCode=0
Jul 10 02:37:35.173941 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:35.173757    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" event={"ID":"395a5a58-da99-4d1d-8a5c-f8028e7d8247","Type":"ContainerDied","Data":"79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811"}
Jul 10 02:37:35.174250 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:35.174232    2557 scope.go:117] "RemoveContainer" containerID="79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811"
Jul 10 02:37:35.175266 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:35.175242    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:37:35.175327 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:35.175313    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:37:35.175367 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:35.175345    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:37:36.180900 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:36.180864    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" event={"ID":"395a5a58-da99-4d1d-8a5c-f8028e7d8247","Type":"ContainerStarted","Data":"204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00"}
Jul 10 02:37:36.752634 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:36.752596    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:36.752878 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:36.752832    2557 patch_prober.go:28] interesting pod/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 10 02:37:36.752930 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:36.752911    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 10 02:37:37.752438 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:37.752397    2557 patch_prober.go:28] interesting pod/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 10 02:37:37.752866 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:37.752471    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 10 02:37:38.752874 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:38.752833    2557 patch_prober.go:28] interesting pod/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 10 02:37:38.753361 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:38.752895    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 10 02:37:39.752441 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:39.752402    2557 patch_prober.go:28] interesting pod/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 10 02:37:39.752629 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:39.752466    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 10 02:37:40.756353 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:40.756317    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:40.756853 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:40.756707    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:37:40.757426 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:37:40.757405    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:38:35.321033 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.320992    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"]
Jul 10 02:38:35.321524 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.321244    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9" podUID="c5dd28d6-41f5-4ee5-ac8e-425b548d2470" containerName="maas-api" containerID="cri-o://4e168ecfe9785355c6afe851b57645aa61ec04c2da5bc418eed2c166b446222b" gracePeriod=30
Jul 10 02:38:35.472321 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.472282    2557 generic.go:356] "Generic (PLEG): container finished" podID="c5dd28d6-41f5-4ee5-ac8e-425b548d2470" containerID="4e168ecfe9785355c6afe851b57645aa61ec04c2da5bc418eed2c166b446222b" exitCode=0
Jul 10 02:38:35.472516 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.472329    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9" event={"ID":"c5dd28d6-41f5-4ee5-ac8e-425b548d2470","Type":"ContainerDied","Data":"4e168ecfe9785355c6afe851b57645aa61ec04c2da5bc418eed2c166b446222b"}
Jul 10 02:38:35.600895 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.600867    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:38:35.724086 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.724053    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptckg\" (UniqueName: \"kubernetes.io/projected/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-kube-api-access-ptckg\") pod \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") "
Jul 10 02:38:35.724086 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.724090    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-openshift-service-ca\") pod \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") "
Jul 10 02:38:35.724335 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.724138    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-maas-api-tls\") pod \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\" (UID: \"c5dd28d6-41f5-4ee5-ac8e-425b548d2470\") "
Jul 10 02:38:35.724497 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.724472    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5dd28d6-41f5-4ee5-ac8e-425b548d2470" (UID: "c5dd28d6-41f5-4ee5-ac8e-425b548d2470"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:38:35.726247 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.726223    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5dd28d6-41f5-4ee5-ac8e-425b548d2470" (UID: "c5dd28d6-41f5-4ee5-ac8e-425b548d2470"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 10 02:38:35.726332 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.726254    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-kube-api-access-ptckg" (OuterVolumeSpecName: "kube-api-access-ptckg") pod "c5dd28d6-41f5-4ee5-ac8e-425b548d2470" (UID: "c5dd28d6-41f5-4ee5-ac8e-425b548d2470"). InnerVolumeSpecName "kube-api-access-ptckg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:38:35.825031 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.824995    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptckg\" (UniqueName: \"kubernetes.io/projected/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-kube-api-access-ptckg\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:35.825031 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.825026    2557 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-openshift-service-ca\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:35.825031 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:35.825037    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5dd28d6-41f5-4ee5-ac8e-425b548d2470-maas-api-tls\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:36.478148 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:36.478124    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"
Jul 10 02:38:36.478603 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:36.478163    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9" event={"ID":"c5dd28d6-41f5-4ee5-ac8e-425b548d2470","Type":"ContainerDied","Data":"fab5e120f4427c451946ce37c34dbeb3731551a341e5b14178e83c2ad0d884df"}
Jul 10 02:38:36.478603 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:36.478224    2557 scope.go:117] "RemoveContainer" containerID="4e168ecfe9785355c6afe851b57645aa61ec04c2da5bc418eed2c166b446222b"
Jul 10 02:38:36.497110 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:36.497083    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"]
Jul 10 02:38:36.502212 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:36.502152    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-bc69ef18-c75d6979f-lqlm9"]
Jul 10 02:38:38.241913 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:38.241881    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5dd28d6-41f5-4ee5-ac8e-425b548d2470" path="/var/lib/kubelet/pods/c5dd28d6-41f5-4ee5-ac8e-425b548d2470/volumes"
Jul 10 02:38:50.104146 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.104062    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"]
Jul 10 02:38:50.104660 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.104274    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy" containerID="cri-o://204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00" gracePeriod=30
Jul 10 02:38:50.501029 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.500948    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"]
Jul 10 02:38:50.501390 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.501375    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5dd28d6-41f5-4ee5-ac8e-425b548d2470" containerName="maas-api"
Jul 10 02:38:50.501471 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.501391    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5dd28d6-41f5-4ee5-ac8e-425b548d2470" containerName="maas-api"
Jul 10 02:38:50.501529 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.501478    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5dd28d6-41f5-4ee5-ac8e-425b548d2470" containerName="maas-api"
Jul 10 02:38:50.506894 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.506868    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.508723 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.508698    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-987cdfe4-openshift-default-dockercfg-6pxwm\""
Jul 10 02:38:50.518120 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.518095    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"]
Jul 10 02:38:50.665552 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.665512    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-credential-socket\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.665552 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.665553    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-token\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.665835 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.665576    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns6t4\" (UniqueName: \"kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-kube-api-access-ns6t4\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.665835 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.665702    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-data\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.665835 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.665783    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istiod-ca-cert\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.666006 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.665839    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-certs\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.666006 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.665866    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-podinfo\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.666006 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.665910    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-socket\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.666006 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.665960    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-envoy\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767088 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.766994    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-envoy\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767088 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767043    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-credential-socket\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767088 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767060    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-token\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767088 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767075    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns6t4\" (UniqueName: \"kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-kube-api-access-ns6t4\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767427 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767106    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-data\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767427 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767162    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istiod-ca-cert\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767427 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767198    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-certs\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767427 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767223    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-podinfo\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767427 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767312    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-socket\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767670 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767573    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-certs\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767670 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767604    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-data\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767737 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767686    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-credential-socket\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767774 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767736    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-socket\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.767993 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.767973    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istiod-ca-cert\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.769386 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.769362    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-envoy\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.769670 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.769652    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-podinfo\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.775982 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.775953    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-token\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.776187 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.776169    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns6t4\" (UniqueName: \"kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-kube-api-access-ns6t4\") pod \"e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") " pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.819389 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.819351    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:50.961952 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.961882    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"]
Jul 10 02:38:50.964316 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:38:50.964287    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6d86be_a5a1_4d66_802f_177ff5ac0d6d.slice/crio-4078a6069b65fac28eaff09236512a7f0eb09f8e12e742827d563fa17a68eb18 WatchSource:0}: Error finding container 4078a6069b65fac28eaff09236512a7f0eb09f8e12e742827d563fa17a68eb18: Status 404 returned error can't find the container with id 4078a6069b65fac28eaff09236512a7f0eb09f8e12e742827d563fa17a68eb18
Jul 10 02:38:50.966420 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.966388    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:38:50.966502 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.966455    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:38:50.966502 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:50.966487    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:38:51.549738 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:51.549703    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" event={"ID":"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d","Type":"ContainerStarted","Data":"93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b"}
Jul 10 02:38:51.550164 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:51.549767    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" event={"ID":"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d","Type":"ContainerStarted","Data":"4078a6069b65fac28eaff09236512a7f0eb09f8e12e742827d563fa17a68eb18"}
Jul 10 02:38:51.569738 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:51.569689    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" podStartSLOduration=1.5696726779999999 podStartE2EDuration="1.569672678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:38:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:38:51.567747633 +0000 UTC m=+1887.884057496" watchObservedRunningTime="2026-07-10 02:38:51.569672678 +0000 UTC m=+1887.885982549"
Jul 10 02:38:51.819888 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:51.819787    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:52.825159 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:52.825126    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:53.560889 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:53.560855    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:53.561847 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:53.561826    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:38:55.353929 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.353906    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:38:55.512754 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.512721    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-envoy\") pod \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") "
Jul 10 02:38:55.512754 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.512756    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-podinfo\") pod \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") "
Jul 10 02:38:55.513035 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.512780    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-socket\") pod \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") "
Jul 10 02:38:55.513035 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.512825    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghqrm\" (UniqueName: \"kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-kube-api-access-ghqrm\") pod \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") "
Jul 10 02:38:55.513035 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.512867    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-credential-socket\") pod \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") "
Jul 10 02:38:55.513035 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.512921    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istiod-ca-cert\") pod \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") "
Jul 10 02:38:55.513035 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.512975    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-certs\") pod \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") "
Jul 10 02:38:55.513035 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.512998    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-token\") pod \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") "
Jul 10 02:38:55.513344 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.513070    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-data\") pod \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\" (UID: \"395a5a58-da99-4d1d-8a5c-f8028e7d8247\") "
Jul 10 02:38:55.513344 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.513124    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "395a5a58-da99-4d1d-8a5c-f8028e7d8247" (UID: "395a5a58-da99-4d1d-8a5c-f8028e7d8247"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:38:55.513344 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.513165    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "395a5a58-da99-4d1d-8a5c-f8028e7d8247" (UID: "395a5a58-da99-4d1d-8a5c-f8028e7d8247"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:38:55.513510 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.513383    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "395a5a58-da99-4d1d-8a5c-f8028e7d8247" (UID: "395a5a58-da99-4d1d-8a5c-f8028e7d8247"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:38:55.513510 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.513420    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:55.513510 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.513439    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-credential-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:55.513713 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.513689    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "395a5a58-da99-4d1d-8a5c-f8028e7d8247" (UID: "395a5a58-da99-4d1d-8a5c-f8028e7d8247"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:38:55.514055 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.514031    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-data" (OuterVolumeSpecName: "istio-data") pod "395a5a58-da99-4d1d-8a5c-f8028e7d8247" (UID: "395a5a58-da99-4d1d-8a5c-f8028e7d8247"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:38:55.515204 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.515183    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-kube-api-access-ghqrm" (OuterVolumeSpecName: "kube-api-access-ghqrm") pod "395a5a58-da99-4d1d-8a5c-f8028e7d8247" (UID: "395a5a58-da99-4d1d-8a5c-f8028e7d8247"). InnerVolumeSpecName "kube-api-access-ghqrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:38:55.515284 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.515187    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "395a5a58-da99-4d1d-8a5c-f8028e7d8247" (UID: "395a5a58-da99-4d1d-8a5c-f8028e7d8247"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:38:55.515393 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.515374    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "395a5a58-da99-4d1d-8a5c-f8028e7d8247" (UID: "395a5a58-da99-4d1d-8a5c-f8028e7d8247"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:38:55.515530 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.515516    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-token" (OuterVolumeSpecName: "istio-token") pod "395a5a58-da99-4d1d-8a5c-f8028e7d8247" (UID: "395a5a58-da99-4d1d-8a5c-f8028e7d8247"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:38:55.571207 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.571175    2557 generic.go:356] "Generic (PLEG): container finished" podID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerID="204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00" exitCode=0
Jul 10 02:38:55.571351 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.571221    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"
Jul 10 02:38:55.571351 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.571252    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" event={"ID":"395a5a58-da99-4d1d-8a5c-f8028e7d8247","Type":"ContainerDied","Data":"204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00"}
Jul 10 02:38:55.571351 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.571303    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6" event={"ID":"395a5a58-da99-4d1d-8a5c-f8028e7d8247","Type":"ContainerDied","Data":"4ea5ccc792375a5f18997080eb7247281b82de706ea0543333f95d7b0f05ff41"}
Jul 10 02:38:55.571351 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.571326    2557 scope.go:117] "RemoveContainer" containerID="204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00"
Jul 10 02:38:55.580115 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.580099    2557 scope.go:117] "RemoveContainer" containerID="79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811"
Jul 10 02:38:55.588152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.588136    2557 scope.go:117] "RemoveContainer" containerID="204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00"
Jul 10 02:38:55.588420 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:38:55.588400    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00\": container with ID starting with 204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00 not found: ID does not exist" containerID="204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00"
Jul 10 02:38:55.588471 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.588429    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00"} err="failed to get container status \"204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00\": rpc error: code = NotFound desc = could not find container \"204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00\": container with ID starting with 204b45f8f99652c5e1c4ecdb965281bb6eea6707eeb2698d0cd894038b324d00 not found: ID does not exist"
Jul 10 02:38:55.588471 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.588446    2557 scope.go:117] "RemoveContainer" containerID="79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811"
Jul 10 02:38:55.588668 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:38:55.588650    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811\": container with ID starting with 79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811 not found: ID does not exist" containerID="79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811"
Jul 10 02:38:55.588721 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.588677    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811"} err="failed to get container status \"79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811\": rpc error: code = NotFound desc = could not find container \"79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811\": container with ID starting with 79e15e3a79ecb752ada480024f90096e70a6e8c3ea7c7315b2891c70b3ef6811 not found: ID does not exist"
Jul 10 02:38:55.596237 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.596216    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"]
Jul 10 02:38:55.603348 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.603326    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bc69ef18-openshift-default-bbcf8fc67-mqlv6"]
Jul 10 02:38:55.614657 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.614637    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istiod-ca-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:55.614657 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.614657    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-workload-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:55.614779 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.614669    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:55.614779 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.614681    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-data\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:55.614779 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.614689    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-envoy\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:55.614779 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.614698    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/395a5a58-da99-4d1d-8a5c-f8028e7d8247-istio-podinfo\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:55.614779 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:55.614706    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghqrm\" (UniqueName: \"kubernetes.io/projected/395a5a58-da99-4d1d-8a5c-f8028e7d8247-kube-api-access-ghqrm\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:38:56.242719 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:38:56.242685    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" path="/var/lib/kubelet/pods/395a5a58-da99-4d1d-8a5c-f8028e7d8247/volumes"
Jul 10 02:39:07.641274 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:07.641240    2557 generic.go:356] "Generic (PLEG): container finished" podID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerID="93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b" exitCode=0
Jul 10 02:39:07.641684 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:07.641314    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" event={"ID":"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d","Type":"ContainerDied","Data":"93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b"}
Jul 10 02:39:07.641730 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:07.641700    2557 scope.go:117] "RemoveContainer" containerID="93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b"
Jul 10 02:39:07.642640 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:07.642615    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:39:07.642691 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:07.642681    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:39:07.642728 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:07.642717    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:39:08.560937 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:08.560897    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:39:08.647934 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:08.647901    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" event={"ID":"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d","Type":"ContainerStarted","Data":"3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644"}
Jul 10 02:39:08.821286 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:08.821167    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:39:08.822310 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:08.822273    2557 patch_prober.go:28] interesting pod/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 10 02:39:08.822452 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:08.822346    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 10 02:39:09.819823 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:09.819758    2557 patch_prober.go:28] interesting pod/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 10 02:39:09.820329 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:09.819847    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 10 02:39:10.820315 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:10.820269    2557 patch_prober.go:28] interesting pod/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 10 02:39:10.820727 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:10.820340    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 10 02:39:11.820362 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:11.820311    2557 patch_prober.go:28] interesting pod/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 10 02:39:11.820763 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:11.820387    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 10 02:39:12.823789 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:12.823754    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:39:12.824214 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:12.824072    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:39:12.824858 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:12.824836    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:39:56.189402 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:56.189365    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"]
Jul 10 02:39:56.189955 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:56.189586    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy" containerID="cri-o://3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644" gracePeriod=30
Jul 10 02:39:57.827256 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:57.827220    2557 patch_prober.go:28] interesting pod/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 02:39:57.827740 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:39:57.827290    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:40:01.453331 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.453252    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:40:01.471187 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.471152    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-socket\") pod \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") "
Jul 10 02:40:01.471370 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.471205    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns6t4\" (UniqueName: \"kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-kube-api-access-ns6t4\") pod \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") "
Jul 10 02:40:01.471370 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.471236    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istiod-ca-cert\") pod \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") "
Jul 10 02:40:01.471370 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.471281    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-token\") pod \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") "
Jul 10 02:40:01.471370 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.471318    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-data\") pod \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") "
Jul 10 02:40:01.471370 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.471342    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-certs\") pod \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") "
Jul 10 02:40:01.471649 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.471410    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-podinfo\") pod \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") "
Jul 10 02:40:01.471649 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.471420    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" (UID: "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:40:01.471649 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.471445    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-credential-socket\") pod \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") "
Jul 10 02:40:01.471649 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.471492    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-envoy\") pod \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\" (UID: \"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d\") "
Jul 10 02:40:01.471881 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.471777    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:40:01.473342 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.473091    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-data" (OuterVolumeSpecName: "istio-data") pod "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" (UID: "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:40:01.473342 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.473111    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" (UID: "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:40:01.473342 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.473263    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" (UID: "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:40:01.473342 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.473318    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" (UID: "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:40:01.476206 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.476169    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-kube-api-access-ns6t4" (OuterVolumeSpecName: "kube-api-access-ns6t4") pod "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" (UID: "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d"). InnerVolumeSpecName "kube-api-access-ns6t4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:40:01.476623 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.476478    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-token" (OuterVolumeSpecName: "istio-token") pod "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" (UID: "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:40:01.476623 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.476581    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" (UID: "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:40:01.476899 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.476618    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" (UID: "4b6d86be-a5a1-4d66-802f-177ff5ac0d6d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:40:01.572466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.572426    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-podinfo\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:40:01.572466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.572460    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-credential-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:40:01.572466 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.572470    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-envoy\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:40:01.572701 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.572479    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns6t4\" (UniqueName: \"kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-kube-api-access-ns6t4\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:40:01.572701 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.572489    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istiod-ca-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:40:01.572701 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.572497    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:40:01.572701 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.572504    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-istio-data\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:40:01.572701 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.572512    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d-workload-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:40:01.899835 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.899782    2557 generic.go:356] "Generic (PLEG): container finished" podID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerID="3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644" exitCode=0
Jul 10 02:40:01.900025 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.899852    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"
Jul 10 02:40:01.900025 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.899871    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" event={"ID":"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d","Type":"ContainerDied","Data":"3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644"}
Jul 10 02:40:01.900025 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.899962    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x" event={"ID":"4b6d86be-a5a1-4d66-802f-177ff5ac0d6d","Type":"ContainerDied","Data":"4078a6069b65fac28eaff09236512a7f0eb09f8e12e742827d563fa17a68eb18"}
Jul 10 02:40:01.900025 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.899988    2557 scope.go:117] "RemoveContainer" containerID="3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644"
Jul 10 02:40:01.913194 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.913173    2557 scope.go:117] "RemoveContainer" containerID="93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b"
Jul 10 02:40:01.920848 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.920830    2557 scope.go:117] "RemoveContainer" containerID="3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644"
Jul 10 02:40:01.921090 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:40:01.921073    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644\": container with ID starting with 3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644 not found: ID does not exist" containerID="3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644"
Jul 10 02:40:01.921189 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.921098    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644"} err="failed to get container status \"3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644\": rpc error: code = NotFound desc = could not find container \"3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644\": container with ID starting with 3431c09d233cc452ff490c2d5fdb2b77dcb376d4ce1650b154cd4ae1936c5644 not found: ID does not exist"
Jul 10 02:40:01.921189 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.921115    2557 scope.go:117] "RemoveContainer" containerID="93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b"
Jul 10 02:40:01.921311 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:40:01.921291    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b\": container with ID starting with 93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b not found: ID does not exist" containerID="93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b"
Jul 10 02:40:01.921349 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.921317    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b"} err="failed to get container status \"93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b\": rpc error: code = NotFound desc = could not find container \"93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b\": container with ID starting with 93c0e5eb7f9feca34345b19240b5e6308117dd4e15e3c8c1ea4835643127663b not found: ID does not exist"
Jul 10 02:40:01.927003 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.926983    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"]
Jul 10 02:40:01.935419 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:01.935395    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-987cdfe4-openshift-default-558c8d4567-78w9x"]
Jul 10 02:40:02.242980 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:40:02.242887    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" path="/var/lib/kubelet/pods/4b6d86be-a5a1-4d66-802f-177ff5ac0d6d/volumes"
Jul 10 02:41:46.645514 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.645429    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"]
Jul 10 02:41:46.645950 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.645856    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy"
Jul 10 02:41:46.645950 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.645876    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy"
Jul 10 02:41:46.645950 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.645903    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy"
Jul 10 02:41:46.645950 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.645921    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy"
Jul 10 02:41:46.646091 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.646035    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy"
Jul 10 02:41:46.646091 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.646047    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy"
Jul 10 02:41:46.646091 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.646061    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy"
Jul 10 02:41:46.649330 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.649313    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.651376 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.651357    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-abb8864d-openshift-default-dockercfg-pplbc\""
Jul 10 02:41:46.668615 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.668576    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"]
Jul 10 02:41:46.688372 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.688341    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istiod-ca-cert\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.688372 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.688374    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-certs\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.688644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.688398    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdg28\" (UniqueName: \"kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-kube-api-access-mdg28\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.688644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.688444    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-envoy\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.688644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.688539    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-token\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.688644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.688580    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-socket\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.688644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.688604    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-data\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.688644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.688627    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-podinfo\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.688905 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.688655    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-credential-socket\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.789397 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.789352    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istiod-ca-cert\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.789397 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.789396    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-certs\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.789672 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.789527    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdg28\" (UniqueName: \"kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-kube-api-access-mdg28\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.789672 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.789571    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-envoy\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.789672 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.789643    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-token\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.789866 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.789684    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-socket\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.789866 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.789710    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-data\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.789866 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.789730    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-certs\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.789866 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.789735    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-podinfo\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.789866 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.789794    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-credential-socket\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.790131 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.790072    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-socket\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.790131 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.790099    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-credential-socket\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.790231 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.790191    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istiod-ca-cert\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.790231 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.790208    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-data\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.791959 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.791932    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-envoy\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.792278 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.792256    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-podinfo\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.800350 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.800324    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-token\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.800562 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.800535    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdg28\" (UniqueName: \"kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-kube-api-access-mdg28\") pod \"e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") " pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:46.959633 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:46.959548    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:47.108756 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:47.108728    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"]
Jul 10 02:41:47.111301 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:41:47.111273    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63a2840e_a7b6_4a5b_987e_e414c746a4a3.slice/crio-cc2108fe84fa0f14332da43d545ed6f38ffd8a353377e524fb30278c78d2ba39 WatchSource:0}: Error finding container cc2108fe84fa0f14332da43d545ed6f38ffd8a353377e524fb30278c78d2ba39: Status 404 returned error can't find the container with id cc2108fe84fa0f14332da43d545ed6f38ffd8a353377e524fb30278c78d2ba39
Jul 10 02:41:47.113797 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:47.113764    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:41:47.113903 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:47.113860    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:41:47.113903 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:47.113889    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:41:47.402195 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:47.402156    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" event={"ID":"63a2840e-a7b6-4a5b-987e-e414c746a4a3","Type":"ContainerStarted","Data":"133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6"}
Jul 10 02:41:47.402376 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:47.402210    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" event={"ID":"63a2840e-a7b6-4a5b-987e-e414c746a4a3","Type":"ContainerStarted","Data":"cc2108fe84fa0f14332da43d545ed6f38ffd8a353377e524fb30278c78d2ba39"}
Jul 10 02:41:47.423710 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:47.423658    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" podStartSLOduration=1.423641218 podStartE2EDuration="1.423641218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:41:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:41:47.42322799 +0000 UTC m=+2063.739537862" watchObservedRunningTime="2026-07-10 02:41:47.423641218 +0000 UTC m=+2063.739951090"
Jul 10 02:41:47.960233 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:47.960189    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:48.964835 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:48.964792    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:49.412653 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:49.412612    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:41:49.413889 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:41:49.413859    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:42:03.480289 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:03.480257    2557 generic.go:356] "Generic (PLEG): container finished" podID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerID="133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6" exitCode=0
Jul 10 02:42:03.480691 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:03.480332    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" event={"ID":"63a2840e-a7b6-4a5b-987e-e414c746a4a3","Type":"ContainerDied","Data":"133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6"}
Jul 10 02:42:03.480789 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:03.480772    2557 scope.go:117] "RemoveContainer" containerID="133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6"
Jul 10 02:42:03.481787 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:03.481761    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:42:03.481882 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:03.481848    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:42:03.481921 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:03.481884    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:42:04.412625 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:04.412576    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:42:04.487397 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:04.487361    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" event={"ID":"63a2840e-a7b6-4a5b-987e-e414c746a4a3","Type":"ContainerStarted","Data":"bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf"}
Jul 10 02:42:04.959872 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:04.959831    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:42:04.961116 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:04.961091    2557 patch_prober.go:28] interesting pod/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 02:42:04.961247 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:04.961138    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 02:42:05.960609 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:05.960570    2557 patch_prober.go:28] interesting pod/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 02:42:05.961025 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:05.960635    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 02:42:06.959951 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:06.959917    2557 patch_prober.go:28] interesting pod/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 02:42:06.960129 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:06.959974    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 02:42:07.960412 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:07.960374    2557 patch_prober.go:28] interesting pod/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 10 02:42:07.960793 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:07.960452    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 10 02:42:08.964014 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:08.963983    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:42:08.964394 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:08.964256    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:42:08.964992 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:08.964976    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:42:24.266795 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:24.266766    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:42:24.277635 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:24.277608    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:42:27.714165 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:27.714127    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"]
Jul 10 02:42:27.714616 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:27.714373    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerName="istio-proxy" containerID="cri-o://bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf" gracePeriod=30
Jul 10 02:42:32.971152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:32.971129    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:42:33.024359 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.024335    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-envoy\") pod \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") "
Jul 10 02:42:33.024528 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.024377    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdg28\" (UniqueName: \"kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-kube-api-access-mdg28\") pod \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") "
Jul 10 02:42:33.024528 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.024394    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-credential-socket\") pod \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") "
Jul 10 02:42:33.024528 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.024420    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-token\") pod \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") "
Jul 10 02:42:33.024528 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.024474    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istiod-ca-cert\") pod \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") "
Jul 10 02:42:33.024528 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.024494    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-podinfo\") pod \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") "
Jul 10 02:42:33.024528 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.024521    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-certs\") pod \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") "
Jul 10 02:42:33.024924 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.024687    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "63a2840e-a7b6-4a5b-987e-e414c746a4a3" (UID: "63a2840e-a7b6-4a5b-987e-e414c746a4a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:33.024924 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.024816    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "63a2840e-a7b6-4a5b-987e-e414c746a4a3" (UID: "63a2840e-a7b6-4a5b-987e-e414c746a4a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:33.024924 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.024864    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-data\") pod \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") "
Jul 10 02:42:33.024924 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.024919    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-socket\") pod \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\" (UID: \"63a2840e-a7b6-4a5b-987e-e414c746a4a3\") "
Jul 10 02:42:33.025180 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.025158    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "63a2840e-a7b6-4a5b-987e-e414c746a4a3" (UID: "63a2840e-a7b6-4a5b-987e-e414c746a4a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:33.025246 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.025220    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:42:33.025246 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.025241    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-credential-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:42:33.025461 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.025434    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "63a2840e-a7b6-4a5b-987e-e414c746a4a3" (UID: "63a2840e-a7b6-4a5b-987e-e414c746a4a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:42:33.025797 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.025722    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-data" (OuterVolumeSpecName: "istio-data") pod "63a2840e-a7b6-4a5b-987e-e414c746a4a3" (UID: "63a2840e-a7b6-4a5b-987e-e414c746a4a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:33.026748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.026718    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-kube-api-access-mdg28" (OuterVolumeSpecName: "kube-api-access-mdg28") pod "63a2840e-a7b6-4a5b-987e-e414c746a4a3" (UID: "63a2840e-a7b6-4a5b-987e-e414c746a4a3"). InnerVolumeSpecName "kube-api-access-mdg28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:42:33.026887 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.026819    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "63a2840e-a7b6-4a5b-987e-e414c746a4a3" (UID: "63a2840e-a7b6-4a5b-987e-e414c746a4a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:42:33.026887 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.026871    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "63a2840e-a7b6-4a5b-987e-e414c746a4a3" (UID: "63a2840e-a7b6-4a5b-987e-e414c746a4a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:42:33.026977 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.026914    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-token" (OuterVolumeSpecName: "istio-token") pod "63a2840e-a7b6-4a5b-987e-e414c746a4a3" (UID: "63a2840e-a7b6-4a5b-987e-e414c746a4a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:42:33.125878 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.125839    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-data\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:42:33.125878 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.125872    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-workload-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:42:33.125878 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.125883    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-envoy\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:42:33.126115 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.125894    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdg28\" (UniqueName: \"kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-kube-api-access-mdg28\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:42:33.126115 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.125904    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:42:33.126115 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.125912    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istiod-ca-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:42:33.126115 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.125920    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63a2840e-a7b6-4a5b-987e-e414c746a4a3-istio-podinfo\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:42:33.645336 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.645299    2557 generic.go:356] "Generic (PLEG): container finished" podID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerID="bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf" exitCode=0
Jul 10 02:42:33.645498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.645362    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" event={"ID":"63a2840e-a7b6-4a5b-987e-e414c746a4a3","Type":"ContainerDied","Data":"bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf"}
Jul 10 02:42:33.645498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.645378    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"
Jul 10 02:42:33.645498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.645411    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml" event={"ID":"63a2840e-a7b6-4a5b-987e-e414c746a4a3","Type":"ContainerDied","Data":"cc2108fe84fa0f14332da43d545ed6f38ffd8a353377e524fb30278c78d2ba39"}
Jul 10 02:42:33.645498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.645435    2557 scope.go:117] "RemoveContainer" containerID="bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf"
Jul 10 02:42:33.654855 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.654794    2557 scope.go:117] "RemoveContainer" containerID="133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6"
Jul 10 02:42:33.662546 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.662528    2557 scope.go:117] "RemoveContainer" containerID="bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf"
Jul 10 02:42:33.662836 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:42:33.662795    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf\": container with ID starting with bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf not found: ID does not exist" containerID="bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf"
Jul 10 02:42:33.662935 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.662854    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf"} err="failed to get container status \"bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf\": rpc error: code = NotFound desc = could not find container \"bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf\": container with ID starting with bd189a498e735aae0ffb56004a4d7c47f2de899a92d5ecd61ff26ecaaedadbcf not found: ID does not exist"
Jul 10 02:42:33.662935 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.662872    2557 scope.go:117] "RemoveContainer" containerID="133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6"
Jul 10 02:42:33.663106 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:42:33.663092    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6\": container with ID starting with 133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6 not found: ID does not exist" containerID="133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6"
Jul 10 02:42:33.663155 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.663109    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6"} err="failed to get container status \"133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6\": rpc error: code = NotFound desc = could not find container \"133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6\": container with ID starting with 133001b5541174c38c5d77949548b292a9bcaa73f99ff946f6839c1ec89fd3f6 not found: ID does not exist"
Jul 10 02:42:33.668523 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.668501    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"]
Jul 10 02:42:33.673343 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:33.673324    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-abb8864d-openshift-default-54868567c9-zp5ml"]
Jul 10 02:42:34.241759 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:34.241726    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" path="/var/lib/kubelet/pods/63a2840e-a7b6-4a5b-987e-e414c746a4a3/volumes"
Jul 10 02:42:48.465912 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.465880    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"]
Jul 10 02:42:48.466295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.466222    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerName="istio-proxy"
Jul 10 02:42:48.466295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.466234    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerName="istio-proxy"
Jul 10 02:42:48.466295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.466247    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy"
Jul 10 02:42:48.466295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.466252    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy"
Jul 10 02:42:48.466295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.466267    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy"
Jul 10 02:42:48.466295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.466273    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b6d86be-a5a1-4d66-802f-177ff5ac0d6d" containerName="istio-proxy"
Jul 10 02:42:48.466479 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.466330    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerName="istio-proxy"
Jul 10 02:42:48.466479 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.466339    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="395a5a58-da99-4d1d-8a5c-f8028e7d8247" containerName="istio-proxy"
Jul 10 02:42:48.468949 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.468931    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.471108 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.471088    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b3d83c99-openshift-default-dockercfg-gvqqz\""
Jul 10 02:42:48.484954 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.484933    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"]
Jul 10 02:42:48.561880 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.561844    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c14b9b8f-058e-489d-8b82-462d611fb8c9-istiod-ca-cert\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.561880 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.561883    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-socket\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.562152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.561924    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-envoy\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.562152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.561940    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-data\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.562152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.562014    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-credential-socket\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.562152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.562065    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-certs\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.562152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.562099    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-podinfo\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.562404 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.562199    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd5vm\" (UniqueName: \"kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-kube-api-access-zd5vm\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.562404 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.562261    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-token\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.662732 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.662686    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-podinfo\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.662732 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.662738    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd5vm\" (UniqueName: \"kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-kube-api-access-zd5vm\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663062 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.662782    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-token\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663062 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.662856    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c14b9b8f-058e-489d-8b82-462d611fb8c9-istiod-ca-cert\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663062 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.662875    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-socket\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663062 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.662905    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-envoy\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663062 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.662931    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-data\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663062 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.662970    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-credential-socket\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663372 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.663081    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-certs\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663481 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.663432    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-certs\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663613 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.663513    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-data\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663613 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.663573    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-socket\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663775 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.663641    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-credential-socket\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.663775 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.663694    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c14b9b8f-058e-489d-8b82-462d611fb8c9-istiod-ca-cert\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.665705 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.665671    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-envoy\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.665824 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.665788    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-podinfo\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.672494 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.672467    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-token\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.672583 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.672494    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd5vm\" (UniqueName: \"kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-kube-api-access-zd5vm\") pod \"e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") " pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.778745 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.778718    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:48.920192 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.920160    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"]
Jul 10 02:42:48.923196 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:42:48.923171    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc14b9b8f_058e_489d_8b82_462d611fb8c9.slice/crio-df57feb2c56ead6e6acc6d5d18fdfecde98cb9af7eb092f601316a4051a8247b WatchSource:0}: Error finding container df57feb2c56ead6e6acc6d5d18fdfecde98cb9af7eb092f601316a4051a8247b: Status 404 returned error can't find the container with id df57feb2c56ead6e6acc6d5d18fdfecde98cb9af7eb092f601316a4051a8247b
Jul 10 02:42:48.925325 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.925293    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:42:48.925413 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.925386    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:42:48.925459 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:48.925444    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:42:49.729854 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:49.729795    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" event={"ID":"c14b9b8f-058e-489d-8b82-462d611fb8c9","Type":"ContainerStarted","Data":"a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89"}
Jul 10 02:42:49.730322 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:49.729904    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" event={"ID":"c14b9b8f-058e-489d-8b82-462d611fb8c9","Type":"ContainerStarted","Data":"df57feb2c56ead6e6acc6d5d18fdfecde98cb9af7eb092f601316a4051a8247b"}
Jul 10 02:42:49.752415 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:49.752368    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" podStartSLOduration=1.752353074 podStartE2EDuration="1.752353074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:42:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:42:49.750597455 +0000 UTC m=+2126.066907324" watchObservedRunningTime="2026-07-10 02:42:49.752353074 +0000 UTC m=+2126.068662945"
Jul 10 02:42:49.779095 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:49.779070    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:49.783621 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:49.783594    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:50.734905 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:50.734873    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:42:50.735974 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:42:50.735952    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:43:05.737120 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:05.737083    2557 patch_prober.go:28] interesting pod/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 02:43:05.737620 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:05.737160    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 02:43:05.812310 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:05.812234    2557 generic.go:356] "Generic (PLEG): container finished" podID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerID="a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89" exitCode=0
Jul 10 02:43:05.812310 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:05.812284    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" event={"ID":"c14b9b8f-058e-489d-8b82-462d611fb8c9","Type":"ContainerDied","Data":"a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89"}
Jul 10 02:43:05.812699 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:05.812684    2557 scope.go:117] "RemoveContainer" containerID="a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89"
Jul 10 02:43:05.813650 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:05.813624    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:43:05.813717 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:05.813694    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:43:05.813763 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:05.813728    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:43:06.821594 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:06.821553    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" event={"ID":"c14b9b8f-058e-489d-8b82-462d611fb8c9","Type":"ContainerStarted","Data":"875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0"}
Jul 10 02:43:07.779247 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:07.779208    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:43:07.779426 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:07.779379    2557 patch_prober.go:28] interesting pod/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 02:43:07.779475 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:07.779426    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 02:43:08.779910 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:08.779876    2557 patch_prober.go:28] interesting pod/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 02:43:08.780304 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:08.779932    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 02:43:09.779923 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:09.779885    2557 patch_prober.go:28] interesting pod/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 10 02:43:09.780312 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:09.779936    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 10 02:43:10.783339 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:10.783308    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:43:10.783760 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:10.783563    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:43:10.784173 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:43:10.784158    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:44:36.774269 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:36.774236    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"]
Jul 10 02:44:36.774739 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:36.774457    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy" containerID="cri-o://875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0" gracePeriod=30
Jul 10 02:44:40.785713 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:40.785677    2557 patch_prober.go:28] interesting pod/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 02:44:40.786198 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:40.785735    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:44:42.042797 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.042737    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:44:42.190012 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.189969    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-certs\") pod \"c14b9b8f-058e-489d-8b82-462d611fb8c9\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") "
Jul 10 02:44:42.190012 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190010    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-token\") pod \"c14b9b8f-058e-489d-8b82-462d611fb8c9\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") "
Jul 10 02:44:42.190277 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190045    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zd5vm\" (UniqueName: \"kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-kube-api-access-zd5vm\") pod \"c14b9b8f-058e-489d-8b82-462d611fb8c9\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") "
Jul 10 02:44:42.190277 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190067    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-podinfo\") pod \"c14b9b8f-058e-489d-8b82-462d611fb8c9\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") "
Jul 10 02:44:42.190277 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190139    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-socket\") pod \"c14b9b8f-058e-489d-8b82-462d611fb8c9\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") "
Jul 10 02:44:42.190277 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190269    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-data\") pod \"c14b9b8f-058e-489d-8b82-462d611fb8c9\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") "
Jul 10 02:44:42.190498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190326    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c14b9b8f-058e-489d-8b82-462d611fb8c9" (UID: "c14b9b8f-058e-489d-8b82-462d611fb8c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:44:42.190498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190343    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c14b9b8f-058e-489d-8b82-462d611fb8c9-istiod-ca-cert\") pod \"c14b9b8f-058e-489d-8b82-462d611fb8c9\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") "
Jul 10 02:44:42.190498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190369    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-envoy\") pod \"c14b9b8f-058e-489d-8b82-462d611fb8c9\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") "
Jul 10 02:44:42.190498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190397    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-credential-socket\") pod \"c14b9b8f-058e-489d-8b82-462d611fb8c9\" (UID: \"c14b9b8f-058e-489d-8b82-462d611fb8c9\") "
Jul 10 02:44:42.190498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190436    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c14b9b8f-058e-489d-8b82-462d611fb8c9" (UID: "c14b9b8f-058e-489d-8b82-462d611fb8c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:44:42.190757 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190672    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c14b9b8f-058e-489d-8b82-462d611fb8c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c14b9b8f-058e-489d-8b82-462d611fb8c9" (UID: "c14b9b8f-058e-489d-8b82-462d611fb8c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:44:42.190757 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190697    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c14b9b8f-058e-489d-8b82-462d611fb8c9" (UID: "c14b9b8f-058e-489d-8b82-462d611fb8c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:44:42.190757 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190758    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:44:42.190978 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190775    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c14b9b8f-058e-489d-8b82-462d611fb8c9-istiod-ca-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:44:42.190978 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.190788    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-workload-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:44:42.191129 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.191104    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "c14b9b8f-058e-489d-8b82-462d611fb8c9" (UID: "c14b9b8f-058e-489d-8b82-462d611fb8c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:44:42.192924 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.192870    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c14b9b8f-058e-489d-8b82-462d611fb8c9" (UID: "c14b9b8f-058e-489d-8b82-462d611fb8c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:44:42.193067 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.192978    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-kube-api-access-zd5vm" (OuterVolumeSpecName: "kube-api-access-zd5vm") pod "c14b9b8f-058e-489d-8b82-462d611fb8c9" (UID: "c14b9b8f-058e-489d-8b82-462d611fb8c9"). InnerVolumeSpecName "kube-api-access-zd5vm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:44:42.193131 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.193116    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c14b9b8f-058e-489d-8b82-462d611fb8c9" (UID: "c14b9b8f-058e-489d-8b82-462d611fb8c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:44:42.193221 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.193205    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "c14b9b8f-058e-489d-8b82-462d611fb8c9" (UID: "c14b9b8f-058e-489d-8b82-462d611fb8c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:44:42.269160 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.269124    2557 generic.go:356] "Generic (PLEG): container finished" podID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerID="875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0" exitCode=0
Jul 10 02:44:42.269312 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.269191    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"
Jul 10 02:44:42.269312 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.269201    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" event={"ID":"c14b9b8f-058e-489d-8b82-462d611fb8c9","Type":"ContainerDied","Data":"875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0"}
Jul 10 02:44:42.269312 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.269254    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j" event={"ID":"c14b9b8f-058e-489d-8b82-462d611fb8c9","Type":"ContainerDied","Data":"df57feb2c56ead6e6acc6d5d18fdfecde98cb9af7eb092f601316a4051a8247b"}
Jul 10 02:44:42.269312 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.269277    2557 scope.go:117] "RemoveContainer" containerID="875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0"
Jul 10 02:44:42.278041 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.278022    2557 scope.go:117] "RemoveContainer" containerID="a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89"
Jul 10 02:44:42.286590 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.286571    2557 scope.go:117] "RemoveContainer" containerID="875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0"
Jul 10 02:44:42.286921 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:44:42.286870    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0\": container with ID starting with 875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0 not found: ID does not exist" containerID="875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0"
Jul 10 02:44:42.287025 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.286933    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0"} err="failed to get container status \"875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0\": rpc error: code = NotFound desc = could not find container \"875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0\": container with ID starting with 875b0261a96da61b0dea04f55919e3cd2b9cb3c20f51d60730095c469bcbfea0 not found: ID does not exist"
Jul 10 02:44:42.287025 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.286959    2557 scope.go:117] "RemoveContainer" containerID="a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89"
Jul 10 02:44:42.287223 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:44:42.287205    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89\": container with ID starting with a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89 not found: ID does not exist" containerID="a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89"
Jul 10 02:44:42.287286 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.287242    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89"} err="failed to get container status \"a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89\": rpc error: code = NotFound desc = could not find container \"a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89\": container with ID starting with a48ac6c7bd1b824e0048b1f29e06d015bfc43b4651d87ab04f52d500d986bc89 not found: ID does not exist"
Jul 10 02:44:42.287442 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.287414    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"]
Jul 10 02:44:42.291985 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.291957    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b3d83c99-openshift-default-67d9dccd87-g5m9j"]
Jul 10 02:44:42.292635 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.292601    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zd5vm\" (UniqueName: \"kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-kube-api-access-zd5vm\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:44:42.292679 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.292648    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-podinfo\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:44:42.292679 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.292669    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-data\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:44:42.292744 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.292689    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-envoy\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:44:42.292744 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.292709    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c14b9b8f-058e-489d-8b82-462d611fb8c9-credential-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:44:42.292744 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:42.292728    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c14b9b8f-058e-489d-8b82-462d611fb8c9-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:44:44.241762 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:44:44.241726    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" path="/var/lib/kubelet/pods/c14b9b8f-058e-489d-8b82-462d611fb8c9/volumes"
Jul 10 02:45:33.597814 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.597767    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"]
Jul 10 02:45:33.598216 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.598121    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerName="istio-proxy"
Jul 10 02:45:33.598216 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.598133    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerName="istio-proxy"
Jul 10 02:45:33.598216 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.598156    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy"
Jul 10 02:45:33.598216 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.598162    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy"
Jul 10 02:45:33.598358 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.598235    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy"
Jul 10 02:45:33.598358 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.598243    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="63a2840e-a7b6-4a5b-987e-e414c746a4a3" containerName="istio-proxy"
Jul 10 02:45:33.598358 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.598250    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy"
Jul 10 02:45:33.601178 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.601159    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.603181 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.603154    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d76849-openshift-default-dockercfg-mhr8g\""
Jul 10 02:45:33.612728 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.612707    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"]
Jul 10 02:45:33.741296 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.741253    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-data\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.741296 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.741301    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853175c1-8eb2-4603-8e0b-8137926aa2d2-istiod-ca-cert\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.741528 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.741338    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-credential-socket\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.741528 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.741357    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-podinfo\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.741528 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.741375    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qjq5\" (UniqueName: \"kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-kube-api-access-9qjq5\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.741528 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.741456    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-token\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.741690 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.741540    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-socket\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.741690 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.741572    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-certs\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.741690 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.741589    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-envoy\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.842727 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.842685    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qjq5\" (UniqueName: \"kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-kube-api-access-9qjq5\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.842954 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.842746    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-token\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.842954 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.842836    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-socket\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.842954 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.842884    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-certs\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.842954 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.842908    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-envoy\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.842954 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.842944    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-data\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.843234 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.842978    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853175c1-8eb2-4603-8e0b-8137926aa2d2-istiod-ca-cert\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.843234 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.843021    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-credential-socket\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.843234 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.843044    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-podinfo\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.843390 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.843269    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-socket\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.843390 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.843343    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-certs\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.843505 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.843429    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-credential-socket\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.843545 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.843500    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-data\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.843788 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.843770    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853175c1-8eb2-4603-8e0b-8137926aa2d2-istiod-ca-cert\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.845179 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.845158    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-envoy\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.845493 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.845470    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-podinfo\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.852168 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.852105    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-token\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.852258 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.852236    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qjq5\" (UniqueName: \"kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-kube-api-access-9qjq5\") pod \"e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") " pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:33.912271 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:33.912230    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:34.051496 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:34.051467    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"]
Jul 10 02:45:34.053110 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:45:34.053082    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853175c1_8eb2_4603_8e0b_8137926aa2d2.slice/crio-b947a1881dd441c447c9f6fc438aa1faf16e8e7d4be6ec9f441f479dcab7ccd3 WatchSource:0}: Error finding container b947a1881dd441c447c9f6fc438aa1faf16e8e7d4be6ec9f441f479dcab7ccd3: Status 404 returned error can't find the container with id b947a1881dd441c447c9f6fc438aa1faf16e8e7d4be6ec9f441f479dcab7ccd3
Jul 10 02:45:34.055474 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:34.055436    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:45:34.055538 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:34.055525    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:45:34.055583 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:34.055565    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:45:34.535546 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:34.535508    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" event={"ID":"853175c1-8eb2-4603-8e0b-8137926aa2d2","Type":"ContainerStarted","Data":"0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58"}
Jul 10 02:45:34.535750 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:34.535573    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" event={"ID":"853175c1-8eb2-4603-8e0b-8137926aa2d2","Type":"ContainerStarted","Data":"b947a1881dd441c447c9f6fc438aa1faf16e8e7d4be6ec9f441f479dcab7ccd3"}
Jul 10 02:45:34.556172 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:34.556114    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" podStartSLOduration=1.556094052 podStartE2EDuration="1.556094052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:34.553947706 +0000 UTC m=+2290.870257593" watchObservedRunningTime="2026-07-10 02:45:34.556094052 +0000 UTC m=+2290.872403923"
Jul 10 02:45:34.913283 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:34.913195    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:35.917351 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:35.917321    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:36.545980 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:36.545937    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:36.546750 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:36.546728    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:42.244187 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.244153    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"]
Jul 10 02:45:42.244698 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.244682    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy"
Jul 10 02:45:42.244754 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.244701    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c14b9b8f-058e-489d-8b82-462d611fb8c9" containerName="istio-proxy"
Jul 10 02:45:42.248408 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.248384    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.250360 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.250335    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-bbcae0-openshift-default-dockercfg-v7xdg\""
Jul 10 02:45:42.254455 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.254428    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"]
Jul 10 02:45:42.421636 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.421602    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-podinfo\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.421856 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.421641    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-envoy\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.421856 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.421661    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-data\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.421856 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.421786    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-certs\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.421856 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.421838    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-token\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.422096 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.421882    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9h56\" (UniqueName: \"kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-kube-api-access-d9h56\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.422096 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.421919    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-socket\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.422096 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.421980    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/477c6275-e149-49d5-9f64-7ade3e75cf11-istiod-ca-cert\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.422096 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.422010    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-credential-socket\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523062 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523033    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-podinfo\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523249 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523077    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-envoy\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523249 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523108    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-data\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523249 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523153    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-certs\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523249 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523179    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-token\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523249 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523211    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9h56\" (UniqueName: \"kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-kube-api-access-d9h56\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523249 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523246    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-socket\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523587 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523300    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/477c6275-e149-49d5-9f64-7ade3e75cf11-istiod-ca-cert\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523587 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523329    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-credential-socket\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523725 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523699    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-data\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523790 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523718    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-credential-socket\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523874 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523838    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-socket\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.523969 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.523948    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-certs\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.524227 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.524204    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/477c6275-e149-49d5-9f64-7ade3e75cf11-istiod-ca-cert\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.525488 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.525467    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-envoy\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.525598 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.525581    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-podinfo\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.531746 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.531718    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-token\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.531861 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.531791    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9h56\" (UniqueName: \"kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-kube-api-access-d9h56\") pod \"e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") " pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.560855 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.560789    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:42.700339 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.700316    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"]
Jul 10 02:45:42.702472 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:45:42.702443    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod477c6275_e149_49d5_9f64_7ade3e75cf11.slice/crio-73b1c1075a7d26f9f2ab6cee4bcd385d2f895e8fcab0d5bf11a6af64c4f4cf28 WatchSource:0}: Error finding container 73b1c1075a7d26f9f2ab6cee4bcd385d2f895e8fcab0d5bf11a6af64c4f4cf28: Status 404 returned error can't find the container with id 73b1c1075a7d26f9f2ab6cee4bcd385d2f895e8fcab0d5bf11a6af64c4f4cf28
Jul 10 02:45:42.705000 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.704970    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:45:42.705084 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.705034    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:45:42.705084 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:42.705062    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:45:43.582986 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:43.582945    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" event={"ID":"477c6275-e149-49d5-9f64-7ade3e75cf11","Type":"ContainerStarted","Data":"ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b"}
Jul 10 02:45:43.583436 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:43.583004    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" event={"ID":"477c6275-e149-49d5-9f64-7ade3e75cf11","Type":"ContainerStarted","Data":"73b1c1075a7d26f9f2ab6cee4bcd385d2f895e8fcab0d5bf11a6af64c4f4cf28"}
Jul 10 02:45:43.604234 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:43.604168    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" podStartSLOduration=1.6041452569999999 podStartE2EDuration="1.604145257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:45:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:45:43.602237097 +0000 UTC m=+2299.918546992" watchObservedRunningTime="2026-07-10 02:45:43.604145257 +0000 UTC m=+2299.920455130"
Jul 10 02:45:44.562003 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:44.561966    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:44.567045 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:44.567015    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:44.588652 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:44.588615    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:44.589601 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:44.589584    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:45:52.588001 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:52.587957    2557 patch_prober.go:28] interesting pod/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 02:45:52.588380 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:52.588013    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 02:45:53.643160 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:53.643124    2557 generic.go:356] "Generic (PLEG): container finished" podID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerID="0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58" exitCode=0
Jul 10 02:45:53.643584 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:53.643199    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" event={"ID":"853175c1-8eb2-4603-8e0b-8137926aa2d2","Type":"ContainerDied","Data":"0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58"}
Jul 10 02:45:53.643627 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:53.643592    2557 scope.go:117] "RemoveContainer" containerID="0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58"
Jul 10 02:45:53.644606 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:53.644574    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:45:53.644677 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:53.644641    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:45:53.644713 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:53.644678    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:45:54.652880 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:54.652843    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" event={"ID":"853175c1-8eb2-4603-8e0b-8137926aa2d2","Type":"ContainerStarted","Data":"a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497"}
Jul 10 02:45:54.913540 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:54.913432    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:54.913820 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:54.913776    2557 patch_prober.go:28] interesting pod/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 02:45:54.913922 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:54.913864    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 02:45:55.912714 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:55.912656    2557 patch_prober.go:28] interesting pod/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 02:45:55.913358 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:55.912728    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 02:45:56.913572 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:56.913534    2557 patch_prober.go:28] interesting pod/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 02:45:56.914168 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:56.914134    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 02:45:57.913117 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:57.913067    2557 patch_prober.go:28] interesting pod/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body=
Jul 10 02:45:57.913337 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:57.913140    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused"
Jul 10 02:45:58.916670 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:58.916638    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:58.917143 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:58.916980    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:45:58.917688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:45:58.917668    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:46:00.629976 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:00.629937    2557 patch_prober.go:28] interesting pod/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 02:46:00.630363 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:00.630039    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 02:46:01.690995 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:01.690960    2557 generic.go:356] "Generic (PLEG): container finished" podID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerID="ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b" exitCode=0
Jul 10 02:46:01.691422 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:01.691037    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" event={"ID":"477c6275-e149-49d5-9f64-7ade3e75cf11","Type":"ContainerDied","Data":"ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b"}
Jul 10 02:46:01.691552 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:01.691536    2557 scope.go:117] "RemoveContainer" containerID="ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b"
Jul 10 02:46:01.692754 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:01.692723    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:46:01.692869 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:01.692836    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:46:01.692937 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:01.692883    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:46:02.700692 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:02.700651    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" event={"ID":"477c6275-e149-49d5-9f64-7ade3e75cf11","Type":"ContainerStarted","Data":"38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803"}
Jul 10 02:46:03.561094 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:03.561065    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:46:03.561282 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:03.561215    2557 patch_prober.go:28] interesting pod/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 10 02:46:03.561282 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:03.561261    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 10 02:46:04.561970 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:04.561934    2557 patch_prober.go:28] interesting pod/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 10 02:46:04.562375 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:04.561987    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 10 02:46:05.561254 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:05.561217    2557 patch_prober.go:28] interesting pod/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 10 02:46:05.561441 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:05.561270    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 10 02:46:06.562200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:06.562161    2557 patch_prober.go:28] interesting pod/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 10 02:46:06.562597 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:06.562218    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 10 02:46:07.564987 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:07.564949    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:46:07.565462 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:07.565230    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:46:07.565907 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:46:07.565886    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:47:24.294717 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:24.294684    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:47:24.307091 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:24.307067    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:47:26.064223 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:26.064189    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"]
Jul 10 02:47:26.064723 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:26.064394    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy" containerID="cri-o://a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497" gracePeriod=30
Jul 10 02:47:28.919143 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:28.919105    2557 patch_prober.go:28] interesting pod/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 02:47:28.919541 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:28.919180    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:47:31.320131 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.320105    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:47:31.447740 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.447641    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-credential-socket\") pod \"853175c1-8eb2-4603-8e0b-8137926aa2d2\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") "
Jul 10 02:47:31.447740 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.447689    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-certs\") pod \"853175c1-8eb2-4603-8e0b-8137926aa2d2\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") "
Jul 10 02:47:31.447740 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.447730    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-podinfo\") pod \"853175c1-8eb2-4603-8e0b-8137926aa2d2\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") "
Jul 10 02:47:31.448050 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.447789    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qjq5\" (UniqueName: \"kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-kube-api-access-9qjq5\") pod \"853175c1-8eb2-4603-8e0b-8137926aa2d2\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") "
Jul 10 02:47:31.448050 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.447839    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-envoy\") pod \"853175c1-8eb2-4603-8e0b-8137926aa2d2\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") "
Jul 10 02:47:31.448050 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.447944    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-token\") pod \"853175c1-8eb2-4603-8e0b-8137926aa2d2\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") "
Jul 10 02:47:31.448050 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.448012    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-socket\") pod \"853175c1-8eb2-4603-8e0b-8137926aa2d2\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") "
Jul 10 02:47:31.448050 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.448026    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "853175c1-8eb2-4603-8e0b-8137926aa2d2" (UID: "853175c1-8eb2-4603-8e0b-8137926aa2d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:31.448050 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.448044    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "853175c1-8eb2-4603-8e0b-8137926aa2d2" (UID: "853175c1-8eb2-4603-8e0b-8137926aa2d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:31.448050 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.448052    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-data\") pod \"853175c1-8eb2-4603-8e0b-8137926aa2d2\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") "
Jul 10 02:47:31.448407 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.448094    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853175c1-8eb2-4603-8e0b-8137926aa2d2-istiod-ca-cert\") pod \"853175c1-8eb2-4603-8e0b-8137926aa2d2\" (UID: \"853175c1-8eb2-4603-8e0b-8137926aa2d2\") "
Jul 10 02:47:31.448407 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.448269    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "853175c1-8eb2-4603-8e0b-8137926aa2d2" (UID: "853175c1-8eb2-4603-8e0b-8137926aa2d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:31.448506 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.448459    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:31.448506 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.448477    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-credential-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:31.448506 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.448491    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-workload-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:31.448657 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.448584    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/853175c1-8eb2-4603-8e0b-8137926aa2d2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "853175c1-8eb2-4603-8e0b-8137926aa2d2" (UID: "853175c1-8eb2-4603-8e0b-8137926aa2d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:47:31.448837 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.448799    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-data" (OuterVolumeSpecName: "istio-data") pod "853175c1-8eb2-4603-8e0b-8137926aa2d2" (UID: "853175c1-8eb2-4603-8e0b-8137926aa2d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:31.450223 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.450194    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "853175c1-8eb2-4603-8e0b-8137926aa2d2" (UID: "853175c1-8eb2-4603-8e0b-8137926aa2d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:31.450223 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.450196    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "853175c1-8eb2-4603-8e0b-8137926aa2d2" (UID: "853175c1-8eb2-4603-8e0b-8137926aa2d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:47:31.450565 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.450537    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-token" (OuterVolumeSpecName: "istio-token") pod "853175c1-8eb2-4603-8e0b-8137926aa2d2" (UID: "853175c1-8eb2-4603-8e0b-8137926aa2d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:31.450670 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.450555    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-kube-api-access-9qjq5" (OuterVolumeSpecName: "kube-api-access-9qjq5") pod "853175c1-8eb2-4603-8e0b-8137926aa2d2" (UID: "853175c1-8eb2-4603-8e0b-8137926aa2d2"). InnerVolumeSpecName "kube-api-access-9qjq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:31.549379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.549343    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-data\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:31.549379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.549375    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/853175c1-8eb2-4603-8e0b-8137926aa2d2-istiod-ca-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:31.549379 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.549385    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-podinfo\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:31.549632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.549394    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qjq5\" (UniqueName: \"kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-kube-api-access-9qjq5\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:31.549632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.549405    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-envoy\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:31.549632 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:31.549412    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/853175c1-8eb2-4603-8e0b-8137926aa2d2-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:32.139195 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.139163    2557 generic.go:356] "Generic (PLEG): container finished" podID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerID="a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497" exitCode=0
Jul 10 02:47:32.139377 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.139206    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" event={"ID":"853175c1-8eb2-4603-8e0b-8137926aa2d2","Type":"ContainerDied","Data":"a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497"}
Jul 10 02:47:32.139377 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.139212    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"
Jul 10 02:47:32.139377 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.139259    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn" event={"ID":"853175c1-8eb2-4603-8e0b-8137926aa2d2","Type":"ContainerDied","Data":"b947a1881dd441c447c9f6fc438aa1faf16e8e7d4be6ec9f441f479dcab7ccd3"}
Jul 10 02:47:32.139377 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.139280    2557 scope.go:117] "RemoveContainer" containerID="a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497"
Jul 10 02:47:32.148649 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.148630    2557 scope.go:117] "RemoveContainer" containerID="0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58"
Jul 10 02:47:32.156570 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.156554    2557 scope.go:117] "RemoveContainer" containerID="a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497"
Jul 10 02:47:32.156838 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:47:32.156814    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497\": container with ID starting with a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497 not found: ID does not exist" containerID="a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497"
Jul 10 02:47:32.156921 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.156850    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497"} err="failed to get container status \"a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497\": rpc error: code = NotFound desc = could not find container \"a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497\": container with ID starting with a8998cfc03875735950e6800c2a81ad7ec2c049a0ec3689cbd84d98b87e82497 not found: ID does not exist"
Jul 10 02:47:32.156921 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.156873    2557 scope.go:117] "RemoveContainer" containerID="0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58"
Jul 10 02:47:32.157095 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:47:32.157078    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58\": container with ID starting with 0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58 not found: ID does not exist" containerID="0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58"
Jul 10 02:47:32.157138 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.157101    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58"} err="failed to get container status \"0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58\": rpc error: code = NotFound desc = could not find container \"0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58\": container with ID starting with 0c4398af3a8a5b95d5abc6002f681fbb511ee4b82a28530b262baa052e1c7c58 not found: ID does not exist"
Jul 10 02:47:32.160927 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.160907    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"]
Jul 10 02:47:32.165155 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.165136    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d76849-openshift-default-78668ccdfc-745wn"]
Jul 10 02:47:32.243161 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:32.243130    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" path="/var/lib/kubelet/pods/853175c1-8eb2-4603-8e0b-8137926aa2d2/volumes"
Jul 10 02:47:52.270895 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:52.270752    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"]
Jul 10 02:47:52.271285 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:52.271047    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy" containerID="cri-o://38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803" gracePeriod=30
Jul 10 02:47:52.568073 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:52.568032    2557 patch_prober.go:28] interesting pod/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 02:47:52.568244 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:52.568104    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:47:53.195301 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.195259    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"]
Jul 10 02:47:53.195638 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.195623    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy"
Jul 10 02:47:53.195717 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.195638    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy"
Jul 10 02:47:53.195717 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.195655    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy"
Jul 10 02:47:53.195717 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.195660    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy"
Jul 10 02:47:53.195904 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.195727    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy"
Jul 10 02:47:53.195904 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.195736    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="853175c1-8eb2-4603-8e0b-8137926aa2d2" containerName="istio-proxy"
Jul 10 02:47:53.200244 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.200222    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.202018 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.201996    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-dbaa7c-openshift-default-dockercfg-6kfn9\""
Jul 10 02:47:53.209847 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.209821    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"]
Jul 10 02:47:53.267566 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.267536    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-istio-token\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.267767 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.267582    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-envoy\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.267767 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.267676    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7dc16ff-9777-4c12-86d0-51663c385369-istiod-ca-cert\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.267767 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.267716    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-socket\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.267767 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.267741    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkpj8\" (UniqueName: \"kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-kube-api-access-pkpj8\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.268035 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.267865    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-credential-socket\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.268035 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.267964    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-certs\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.268035 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.267983    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7dc16ff-9777-4c12-86d0-51663c385369-istio-podinfo\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.268035 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.268014    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-data\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.368942 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.368899    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-istio-token\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369403 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.368948    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-envoy\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369403 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.368995    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7dc16ff-9777-4c12-86d0-51663c385369-istiod-ca-cert\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369403 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.369022    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-socket\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369403 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.369049    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkpj8\" (UniqueName: \"kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-kube-api-access-pkpj8\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369403 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.369199    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-credential-socket\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369403 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.369252    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-certs\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369403 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.369277    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7dc16ff-9777-4c12-86d0-51663c385369-istio-podinfo\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369403 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.369329    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-data\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369852 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.369509    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-socket\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369852 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.369549    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-credential-socket\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369852 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.369627    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-certs\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369852 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.369691    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-data\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.369852 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.369759    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7dc16ff-9777-4c12-86d0-51663c385369-istiod-ca-cert\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.371377 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.371354    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-envoy\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.371737 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.371719    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7dc16ff-9777-4c12-86d0-51663c385369-istio-podinfo\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.378445 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.378412    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkpj8\" (UniqueName: \"kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-kube-api-access-pkpj8\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.378561 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.378512    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-istio-token\") pod \"e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") " pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.512079 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.511976    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:53.650236 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.650210    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"]
Jul 10 02:47:53.652303 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:47:53.652275    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7dc16ff_9777_4c12_86d0_51663c385369.slice/crio-3f2b0e86ca3ac7ccea1f594fff7be16ac1089a76bee2f1ecbf3d64beb7a64054 WatchSource:0}: Error finding container 3f2b0e86ca3ac7ccea1f594fff7be16ac1089a76bee2f1ecbf3d64beb7a64054: Status 404 returned error can't find the container with id 3f2b0e86ca3ac7ccea1f594fff7be16ac1089a76bee2f1ecbf3d64beb7a64054
Jul 10 02:47:53.654583 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.654539    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:47:53.654664 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.654640    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:47:53.654707 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:53.654685    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:47:54.251421 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:54.251384    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" event={"ID":"b7dc16ff-9777-4c12-86d0-51663c385369","Type":"ContainerStarted","Data":"1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda"}
Jul 10 02:47:54.251594 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:54.251438    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" event={"ID":"b7dc16ff-9777-4c12-86d0-51663c385369","Type":"ContainerStarted","Data":"3f2b0e86ca3ac7ccea1f594fff7be16ac1089a76bee2f1ecbf3d64beb7a64054"}
Jul 10 02:47:54.275304 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:54.275245    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" podStartSLOduration=1.275225263 podStartE2EDuration="1.275225263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:47:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:47:54.272062719 +0000 UTC m=+2430.588372617" watchObservedRunningTime="2026-07-10 02:47:54.275225263 +0000 UTC m=+2430.591535138"
Jul 10 02:47:54.512724 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:54.512636    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:54.517321 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:54.517296    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:55.256773 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:55.256737    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:55.257655 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:55.257633    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:47:57.531406 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.531347    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:47:57.610772 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.610740    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-podinfo\") pod \"477c6275-e149-49d5-9f64-7ade3e75cf11\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") "
Jul 10 02:47:57.610945 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.610790    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-credential-socket\") pod \"477c6275-e149-49d5-9f64-7ade3e75cf11\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") "
Jul 10 02:47:57.610945 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.610846    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-certs\") pod \"477c6275-e149-49d5-9f64-7ade3e75cf11\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") "
Jul 10 02:47:57.610945 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.610865    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-token\") pod \"477c6275-e149-49d5-9f64-7ade3e75cf11\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") "
Jul 10 02:47:57.611098 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611033    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9h56\" (UniqueName: \"kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-kube-api-access-d9h56\") pod \"477c6275-e149-49d5-9f64-7ade3e75cf11\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") "
Jul 10 02:47:57.611155 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611116    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "477c6275-e149-49d5-9f64-7ade3e75cf11" (UID: "477c6275-e149-49d5-9f64-7ade3e75cf11"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:57.611155 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611120    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-envoy\") pod \"477c6275-e149-49d5-9f64-7ade3e75cf11\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") "
Jul 10 02:47:57.611264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611147    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "477c6275-e149-49d5-9f64-7ade3e75cf11" (UID: "477c6275-e149-49d5-9f64-7ade3e75cf11"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:57.611264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611169    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-socket\") pod \"477c6275-e149-49d5-9f64-7ade3e75cf11\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") "
Jul 10 02:47:57.611264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611200    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-data\") pod \"477c6275-e149-49d5-9f64-7ade3e75cf11\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") "
Jul 10 02:47:57.611423 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611283    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/477c6275-e149-49d5-9f64-7ade3e75cf11-istiod-ca-cert\") pod \"477c6275-e149-49d5-9f64-7ade3e75cf11\" (UID: \"477c6275-e149-49d5-9f64-7ade3e75cf11\") "
Jul 10 02:47:57.611423 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611369    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "477c6275-e149-49d5-9f64-7ade3e75cf11" (UID: "477c6275-e149-49d5-9f64-7ade3e75cf11"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:57.611700 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611633    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-credential-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:57.611700 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611636    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/477c6275-e149-49d5-9f64-7ade3e75cf11-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "477c6275-e149-49d5-9f64-7ade3e75cf11" (UID: "477c6275-e149-49d5-9f64-7ade3e75cf11"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:47:57.611700 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611658    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:57.611700 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.611673    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-workload-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:57.612103 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.612075    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-data" (OuterVolumeSpecName: "istio-data") pod "477c6275-e149-49d5-9f64-7ade3e75cf11" (UID: "477c6275-e149-49d5-9f64-7ade3e75cf11"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:57.613238 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.613206    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "477c6275-e149-49d5-9f64-7ade3e75cf11" (UID: "477c6275-e149-49d5-9f64-7ade3e75cf11"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:47:57.613359 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.613242    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "477c6275-e149-49d5-9f64-7ade3e75cf11" (UID: "477c6275-e149-49d5-9f64-7ade3e75cf11"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:47:57.613359 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.613252    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-token" (OuterVolumeSpecName: "istio-token") pod "477c6275-e149-49d5-9f64-7ade3e75cf11" (UID: "477c6275-e149-49d5-9f64-7ade3e75cf11"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:57.613359 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.613314    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-kube-api-access-d9h56" (OuterVolumeSpecName: "kube-api-access-d9h56") pod "477c6275-e149-49d5-9f64-7ade3e75cf11" (UID: "477c6275-e149-49d5-9f64-7ade3e75cf11"). InnerVolumeSpecName "kube-api-access-d9h56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:47:57.713067 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.713033    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:57.713067 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.713062    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9h56\" (UniqueName: \"kubernetes.io/projected/477c6275-e149-49d5-9f64-7ade3e75cf11-kube-api-access-d9h56\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:57.713067 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.713075    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-envoy\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:57.713295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.713088    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-data\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:57.713295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.713099    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/477c6275-e149-49d5-9f64-7ade3e75cf11-istiod-ca-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:57.713295 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:57.713109    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/477c6275-e149-49d5-9f64-7ade3e75cf11-istio-podinfo\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:47:58.273284 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.273253    2557 generic.go:356] "Generic (PLEG): container finished" podID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerID="38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803" exitCode=0
Jul 10 02:47:58.273414 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.273304    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"
Jul 10 02:47:58.273414 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.273309    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" event={"ID":"477c6275-e149-49d5-9f64-7ade3e75cf11","Type":"ContainerDied","Data":"38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803"}
Jul 10 02:47:58.273414 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.273352    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg" event={"ID":"477c6275-e149-49d5-9f64-7ade3e75cf11","Type":"ContainerDied","Data":"73b1c1075a7d26f9f2ab6cee4bcd385d2f895e8fcab0d5bf11a6af64c4f4cf28"}
Jul 10 02:47:58.273414 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.273373    2557 scope.go:117] "RemoveContainer" containerID="38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803"
Jul 10 02:47:58.282564 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.282543    2557 scope.go:117] "RemoveContainer" containerID="ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b"
Jul 10 02:47:58.290267 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.290246    2557 scope.go:117] "RemoveContainer" containerID="38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803"
Jul 10 02:47:58.290549 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:47:58.290528    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803\": container with ID starting with 38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803 not found: ID does not exist" containerID="38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803"
Jul 10 02:47:58.290593 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.290560    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803"} err="failed to get container status \"38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803\": rpc error: code = NotFound desc = could not find container \"38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803\": container with ID starting with 38617f772e302bc463e0154d494f368add5b310148b2e0d8a33f288e2eea1803 not found: ID does not exist"
Jul 10 02:47:58.290593 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.290578    2557 scope.go:117] "RemoveContainer" containerID="ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b"
Jul 10 02:47:58.290840 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:47:58.290818    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b\": container with ID starting with ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b not found: ID does not exist" containerID="ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b"
Jul 10 02:47:58.290928 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.290845    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b"} err="failed to get container status \"ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b\": rpc error: code = NotFound desc = could not find container \"ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b\": container with ID starting with ac7a5286320c0ba55a73c510f8881932383788428f2aae733934071fe049db4b not found: ID does not exist"
Jul 10 02:47:58.293408 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.293385    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"]
Jul 10 02:47:58.298102 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:47:58.298080    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-bbcae0-openshift-default-584fc984f-j7tdg"]
Jul 10 02:48:00.244241 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:00.244195    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" path="/var/lib/kubelet/pods/477c6275-e149-49d5-9f64-7ade3e75cf11/volumes"
Jul 10 02:48:01.536612 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.536580    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"]
Jul 10 02:48:01.537096 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.537079    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy"
Jul 10 02:48:01.537142 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.537100    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy"
Jul 10 02:48:01.537142 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.537119    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy"
Jul 10 02:48:01.537142 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.537129    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy"
Jul 10 02:48:01.537237 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.537226    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy"
Jul 10 02:48:01.540693 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.540666    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.542563 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.542543    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-20e0d1-openshift-default-dockercfg-7zlb6\""
Jul 10 02:48:01.552591 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.552572    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"]
Jul 10 02:48:01.648922 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.648883    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-token\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.649090 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.648938    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istiod-ca-cert\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.649090 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.648968    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjgfv\" (UniqueName: \"kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-kube-api-access-jjgfv\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.649090 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.649051    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-certs\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.649220 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.649118    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-data\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.649220 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.649156    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-credential-socket\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.649220 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.649183    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-socket\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.649322 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.649269    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-podinfo\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.649322 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.649301    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-envoy\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.750213 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750178    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-data\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.750213 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750221    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-credential-socket\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.750475 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750239    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-socket\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.750475 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750375    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-podinfo\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.750475 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750419    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-envoy\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.750475 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750458    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-token\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.750703 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750499    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istiod-ca-cert\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.750703 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750534    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjgfv\" (UniqueName: \"kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-kube-api-access-jjgfv\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.750703 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750573    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-certs\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.750703 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750604    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-credential-socket\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.750703 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750634    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-socket\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.751014 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750700    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-data\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.751014 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.750953    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-certs\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.751226 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.751207    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istiod-ca-cert\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.752725 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.752702    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-envoy\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.753047 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.753027    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-podinfo\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.759276 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.759251    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjgfv\" (UniqueName: \"kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-kube-api-access-jjgfv\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.759675 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.759657    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-token\") pod \"e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") " pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:01.851637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:01.851564    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:02.015672 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:02.015587    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"]
Jul 10 02:48:02.023846 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:48:02.023072    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb7b55f4_0dc4_46d1_b2ce_fb1d47895676.slice/crio-ed115a4749c2ca513861de50bc32183d49e4d99645d4da264ecc8db70ff7db56 WatchSource:0}: Error finding container ed115a4749c2ca513861de50bc32183d49e4d99645d4da264ecc8db70ff7db56: Status 404 returned error can't find the container with id ed115a4749c2ca513861de50bc32183d49e4d99645d4da264ecc8db70ff7db56
Jul 10 02:48:02.025656 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:02.025621    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:48:02.025760 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:02.025707    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:48:02.025760 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:02.025751    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:48:02.296418 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:02.296369    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" event={"ID":"db7b55f4-0dc4-46d1-b2ce-fb1d47895676","Type":"ContainerStarted","Data":"228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc"}
Jul 10 02:48:02.296611 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:02.296439    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" event={"ID":"db7b55f4-0dc4-46d1-b2ce-fb1d47895676","Type":"ContainerStarted","Data":"ed115a4749c2ca513861de50bc32183d49e4d99645d4da264ecc8db70ff7db56"}
Jul 10 02:48:02.319159 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:02.319094    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" podStartSLOduration=1.319072663 podStartE2EDuration="1.319072663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:48:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:48:02.315491166 +0000 UTC m=+2438.631801041" watchObservedRunningTime="2026-07-10 02:48:02.319072663 +0000 UTC m=+2438.635382536"
Jul 10 02:48:02.852667 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:02.852630    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:03.857328 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:03.857292    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:04.312077 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:04.312044    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:04.315075 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:04.315049    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:09.340298 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:09.340269    2557 generic.go:356] "Generic (PLEG): container finished" podID="b7dc16ff-9777-4c12-86d0-51663c385369" containerID="1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda" exitCode=0
Jul 10 02:48:09.340717 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:09.340317    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" event={"ID":"b7dc16ff-9777-4c12-86d0-51663c385369","Type":"ContainerDied","Data":"1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda"}
Jul 10 02:48:09.340717 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:09.340708    2557 scope.go:117] "RemoveContainer" containerID="1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda"
Jul 10 02:48:09.341693 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:09.341667    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:48:09.341754 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:09.341732    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:48:09.341792 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:09.341763    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:48:10.257684 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:10.257647    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:48:10.347878 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:10.347842    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" event={"ID":"b7dc16ff-9777-4c12-86d0-51663c385369","Type":"ContainerStarted","Data":"c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd"}
Jul 10 02:48:10.512628 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:10.512520    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:48:10.514004 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:10.513971    2557 patch_prober.go:28] interesting pod/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 10 02:48:10.514137 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:10.514046    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 10 02:48:11.513000 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:11.512956    2557 patch_prober.go:28] interesting pod/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 10 02:48:11.513498 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:11.513036    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 10 02:48:12.513147 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:12.513104    2557 patch_prober.go:28] interesting pod/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 10 02:48:12.513763 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:12.513728    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 10 02:48:13.513000 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:13.512960    2557 patch_prober.go:28] interesting pod/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 10 02:48:13.513324 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:13.513293    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 10 02:48:14.517776 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:14.517734    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:48:14.518268 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:14.518245    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:48:14.518951 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:14.518934    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:48:20.354033 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:20.353985    2557 patch_prober.go:28] interesting pod/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 10 02:48:20.354524 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:20.354056    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 10 02:48:22.410579 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:22.410547    2557 generic.go:356] "Generic (PLEG): container finished" podID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerID="228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc" exitCode=0
Jul 10 02:48:22.411080 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:22.410618    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" event={"ID":"db7b55f4-0dc4-46d1-b2ce-fb1d47895676","Type":"ContainerDied","Data":"228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc"}
Jul 10 02:48:22.411080 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:22.411006    2557 scope.go:117] "RemoveContainer" containerID="228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc"
Jul 10 02:48:22.411937 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:22.411911    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:48:22.412039 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:22.411977    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:48:22.412039 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:22.412009    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:48:23.420445 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:23.420395    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" event={"ID":"db7b55f4-0dc4-46d1-b2ce-fb1d47895676","Type":"ContainerStarted","Data":"cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846"}
Jul 10 02:48:23.852382 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:23.852344    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:23.852603 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:23.852575    2557 patch_prober.go:28] interesting pod/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 10 02:48:23.852685 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:23.852641    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 10 02:48:24.852535 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:24.852495    2557 patch_prober.go:28] interesting pod/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 10 02:48:24.852985 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:24.852563    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 10 02:48:25.852768 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:25.852733    2557 patch_prober.go:28] interesting pod/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 10 02:48:25.853200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:25.852787    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 10 02:48:26.852825 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:26.852765    2557 patch_prober.go:28] interesting pod/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body=
Jul 10 02:48:26.853220 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:26.852838    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused"
Jul 10 02:48:27.856752 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:27.856724    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:27.857141 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:27.857079    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:27.857755 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:27.857733    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:48:42.856042 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:42.856009    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"]
Jul 10 02:48:42.856617 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:42.856213    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy" containerID="cri-o://c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd" gracePeriod=30
Jul 10 02:48:44.520154 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:44.520110    2557 patch_prober.go:28] interesting pod/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 02:48:44.520520 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:44.520186    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:48:48.120458 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.117000    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:48:48.192839 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.192786    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-credential-socket\") pod \"b7dc16ff-9777-4c12-86d0-51663c385369\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") "
Jul 10 02:48:48.193044 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.192862    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-socket\") pod \"b7dc16ff-9777-4c12-86d0-51663c385369\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") "
Jul 10 02:48:48.193044 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.192906    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-certs\") pod \"b7dc16ff-9777-4c12-86d0-51663c385369\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") "
Jul 10 02:48:48.193044 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.192946    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-envoy\") pod \"b7dc16ff-9777-4c12-86d0-51663c385369\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") "
Jul 10 02:48:48.193044 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.192980    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkpj8\" (UniqueName: \"kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-kube-api-access-pkpj8\") pod \"b7dc16ff-9777-4c12-86d0-51663c385369\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") "
Jul 10 02:48:48.193044 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.193033    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-data\") pod \"b7dc16ff-9777-4c12-86d0-51663c385369\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") "
Jul 10 02:48:48.193341 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.193088    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7dc16ff-9777-4c12-86d0-51663c385369-istiod-ca-cert\") pod \"b7dc16ff-9777-4c12-86d0-51663c385369\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") "
Jul 10 02:48:48.193341 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.193123    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-istio-token\") pod \"b7dc16ff-9777-4c12-86d0-51663c385369\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") "
Jul 10 02:48:48.193341 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.193113    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b7dc16ff-9777-4c12-86d0-51663c385369" (UID: "b7dc16ff-9777-4c12-86d0-51663c385369"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:48:48.193341 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.193150    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7dc16ff-9777-4c12-86d0-51663c385369-istio-podinfo\") pod \"b7dc16ff-9777-4c12-86d0-51663c385369\" (UID: \"b7dc16ff-9777-4c12-86d0-51663c385369\") "
Jul 10 02:48:48.193341 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.193176    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b7dc16ff-9777-4c12-86d0-51663c385369" (UID: "b7dc16ff-9777-4c12-86d0-51663c385369"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:48:48.193612 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.193467    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:48:48.193612 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.193485    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-credential-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:48:48.193612 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.193481    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7dc16ff-9777-4c12-86d0-51663c385369-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b7dc16ff-9777-4c12-86d0-51663c385369" (UID: "b7dc16ff-9777-4c12-86d0-51663c385369"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:48:48.193612 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.193577    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b7dc16ff-9777-4c12-86d0-51663c385369" (UID: "b7dc16ff-9777-4c12-86d0-51663c385369"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:48:48.194417 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.194385    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-data" (OuterVolumeSpecName: "istio-data") pod "b7dc16ff-9777-4c12-86d0-51663c385369" (UID: "b7dc16ff-9777-4c12-86d0-51663c385369"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:48:48.195399 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.195373    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-istio-token" (OuterVolumeSpecName: "istio-token") pod "b7dc16ff-9777-4c12-86d0-51663c385369" (UID: "b7dc16ff-9777-4c12-86d0-51663c385369"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:48:48.195819 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.195704    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-kube-api-access-pkpj8" (OuterVolumeSpecName: "kube-api-access-pkpj8") pod "b7dc16ff-9777-4c12-86d0-51663c385369" (UID: "b7dc16ff-9777-4c12-86d0-51663c385369"). InnerVolumeSpecName "kube-api-access-pkpj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:48:48.195819 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.195767    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b7dc16ff-9777-4c12-86d0-51663c385369" (UID: "b7dc16ff-9777-4c12-86d0-51663c385369"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:48:48.195819 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.195772    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7dc16ff-9777-4c12-86d0-51663c385369-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b7dc16ff-9777-4c12-86d0-51663c385369" (UID: "b7dc16ff-9777-4c12-86d0-51663c385369"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:48:48.294386 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.294206    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-envoy\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:48:48.294386 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.294237    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkpj8\" (UniqueName: \"kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-kube-api-access-pkpj8\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:48:48.294386 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.294252    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-istio-data\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:48:48.294386 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.294268    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7dc16ff-9777-4c12-86d0-51663c385369-istiod-ca-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:48:48.294386 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.294280    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7dc16ff-9777-4c12-86d0-51663c385369-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:48:48.294386 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.294292    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7dc16ff-9777-4c12-86d0-51663c385369-istio-podinfo\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:48:48.294386 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.294305    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7dc16ff-9777-4c12-86d0-51663c385369-workload-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:48:48.544464 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.544432    2557 generic.go:356] "Generic (PLEG): container finished" podID="b7dc16ff-9777-4c12-86d0-51663c385369" containerID="c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd" exitCode=0
Jul 10 02:48:48.544637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.544484    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"
Jul 10 02:48:48.544637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.544520    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" event={"ID":"b7dc16ff-9777-4c12-86d0-51663c385369","Type":"ContainerDied","Data":"c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd"}
Jul 10 02:48:48.544637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.544578    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d" event={"ID":"b7dc16ff-9777-4c12-86d0-51663c385369","Type":"ContainerDied","Data":"3f2b0e86ca3ac7ccea1f594fff7be16ac1089a76bee2f1ecbf3d64beb7a64054"}
Jul 10 02:48:48.544637 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.544608    2557 scope.go:117] "RemoveContainer" containerID="c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd"
Jul 10 02:48:48.553271 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.553253    2557 scope.go:117] "RemoveContainer" containerID="1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda"
Jul 10 02:48:48.561606 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.561588    2557 scope.go:117] "RemoveContainer" containerID="c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd"
Jul 10 02:48:48.561894 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:48:48.561875    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd\": container with ID starting with c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd not found: ID does not exist" containerID="c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd"
Jul 10 02:48:48.561952 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.561903    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd"} err="failed to get container status \"c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd\": rpc error: code = NotFound desc = could not find container \"c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd\": container with ID starting with c177eb755f03e815d65d8788fffb2720bc8bf5c3d64a43df9988eae214e60bcd not found: ID does not exist"
Jul 10 02:48:48.561952 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.561921    2557 scope.go:117] "RemoveContainer" containerID="1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda"
Jul 10 02:48:48.562170 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:48:48.562153    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda\": container with ID starting with 1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda not found: ID does not exist" containerID="1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda"
Jul 10 02:48:48.562225 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.562179    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda"} err="failed to get container status \"1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda\": rpc error: code = NotFound desc = could not find container \"1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda\": container with ID starting with 1ad324be63c5b7bbf88e9a782330c161704e18cef4c1b6ebbb26f83071831fda not found: ID does not exist"
Jul 10 02:48:48.569107 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.569085    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"]
Jul 10 02:48:48.577523 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:48.577494    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-dbaa7c-openshift-default-5f4bbd75d7-bxj2d"]
Jul 10 02:48:50.243186 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:48:50.243152    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" path="/var/lib/kubelet/pods/b7dc16ff-9777-4c12-86d0-51663c385369/volumes"
Jul 10 02:49:09.066929 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:09.066891    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"]
Jul 10 02:49:09.067352 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:09.067162    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy" containerID="cri-o://cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846" gracePeriod=30
Jul 10 02:49:10.012838 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.012791    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"]
Jul 10 02:49:10.013192 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.013180    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy"
Jul 10 02:49:10.013241 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.013194    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy"
Jul 10 02:49:10.013241 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.013202    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy"
Jul 10 02:49:10.013241 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.013207    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy"
Jul 10 02:49:10.013334 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.013263    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="477c6275-e149-49d5-9f64-7ade3e75cf11" containerName="istio-proxy"
Jul 10 02:49:10.013334 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.013273    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy"
Jul 10 02:49:10.013334 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.013280    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7dc16ff-9777-4c12-86d0-51663c385369" containerName="istio-proxy"
Jul 10 02:49:10.016231 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.016206    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.018139 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.018114    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-d5b98a-openshift-default-dockercfg-jqx9z\""
Jul 10 02:49:10.029577 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.029553    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"]
Jul 10 02:49:10.086795 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.086755    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-envoy\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.087200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.086796    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-podinfo\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.087200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.086862    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istiod-ca-cert\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.087200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.086933    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-socket\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.087200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.086954    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-credential-socket\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.087200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.086972    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-data\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.087200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.086988    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m65p2\" (UniqueName: \"kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-kube-api-access-m65p2\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.087200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.087073    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-token\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.087200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.087139    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-certs\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.187758 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.187719    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istiod-ca-cert\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.187964 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.187773    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-socket\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.187964 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.187887    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-credential-socket\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.187964 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.187923    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-data\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.187964 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.187946    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m65p2\" (UniqueName: \"kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-kube-api-access-m65p2\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.188157 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.187998    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-token\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.188157 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.188071    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-certs\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.188157 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.188135    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-envoy\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.188157 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.188138    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-socket\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.188340 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.188163    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-podinfo\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.188340 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.188197    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-credential-socket\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.188340 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.188263    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-data\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.188478 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.188426    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-certs\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.188568 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.188541    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istiod-ca-cert\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.190418 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.190388    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-envoy\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.190507 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.190490    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-podinfo\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.197021 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.196993    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m65p2\" (UniqueName: \"kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-kube-api-access-m65p2\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.197189 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.197166    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-token\") pod \"e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") " pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.327136 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.327098    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:10.474826 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.474780    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"]
Jul 10 02:49:10.477416 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:49:10.477388    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc8bdf6_924e_4fa4_a7a8_982ae5caf697.slice/crio-821cf3bb4b5faec093b78b4f012354c3848b17bd11ad3d3a5433cd6ada290058 WatchSource:0}: Error finding container 821cf3bb4b5faec093b78b4f012354c3848b17bd11ad3d3a5433cd6ada290058: Status 404 returned error can't find the container with id 821cf3bb4b5faec093b78b4f012354c3848b17bd11ad3d3a5433cd6ada290058
Jul 10 02:49:10.479504 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.479472    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:10.479594 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.479535    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:10.479594 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.479568    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:10.660591 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.660489    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" event={"ID":"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697","Type":"ContainerStarted","Data":"a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273"}
Jul 10 02:49:10.660591 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.660538    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" event={"ID":"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697","Type":"ContainerStarted","Data":"821cf3bb4b5faec093b78b4f012354c3848b17bd11ad3d3a5433cd6ada290058"}
Jul 10 02:49:10.680742 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:10.680680    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" podStartSLOduration=1.680663631 podStartE2EDuration="1.680663631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:49:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:49:10.678520018 +0000 UTC m=+2506.994829890" watchObservedRunningTime="2026-07-10 02:49:10.680663631 +0000 UTC m=+2506.996973503"
Jul 10 02:49:11.327514 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:11.327478    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:11.328867 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:11.328835    2557 patch_prober.go:28] interesting pod/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 10 02:49:11.328994 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:11.328889    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 10 02:49:12.331968 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:12.331936    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:12.670787 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:12.670693    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:12.671843 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:12.671793    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:12.859200 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:12.859164    2557 patch_prober.go:28] interesting pod/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 02:49:12.859364 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:12.859221    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:49:14.335755 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.335690    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:49:14.427082 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427049    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-credential-socket\") pod \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") "
Jul 10 02:49:14.427261 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427094    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-token\") pod \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") "
Jul 10 02:49:14.427261 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427130    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjgfv\" (UniqueName: \"kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-kube-api-access-jjgfv\") pod \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") "
Jul 10 02:49:14.427261 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427145    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-data\") pod \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") "
Jul 10 02:49:14.427420 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427262    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-certs\") pod \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") "
Jul 10 02:49:14.427420 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427292    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-socket\") pod \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") "
Jul 10 02:49:14.427420 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427327    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istiod-ca-cert\") pod \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") "
Jul 10 02:49:14.427420 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427367    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-envoy\") pod \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") "
Jul 10 02:49:14.427420 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427410    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-podinfo\") pod \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\" (UID: \"db7b55f4-0dc4-46d1-b2ce-fb1d47895676\") "
Jul 10 02:49:14.427671 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427418    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db7b55f4-0dc4-46d1-b2ce-fb1d47895676" (UID: "db7b55f4-0dc4-46d1-b2ce-fb1d47895676"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:49:14.427671 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427514    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db7b55f4-0dc4-46d1-b2ce-fb1d47895676" (UID: "db7b55f4-0dc4-46d1-b2ce-fb1d47895676"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:49:14.427671 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427601    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db7b55f4-0dc4-46d1-b2ce-fb1d47895676" (UID: "db7b55f4-0dc4-46d1-b2ce-fb1d47895676"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:49:14.427847 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427775    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-credential-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:49:14.427847 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427796    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:49:14.427941 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.427846    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-workload-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:49:14.428096 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.428072    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db7b55f4-0dc4-46d1-b2ce-fb1d47895676" (UID: "db7b55f4-0dc4-46d1-b2ce-fb1d47895676"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:49:14.428722 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.428701    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-data" (OuterVolumeSpecName: "istio-data") pod "db7b55f4-0dc4-46d1-b2ce-fb1d47895676" (UID: "db7b55f4-0dc4-46d1-b2ce-fb1d47895676"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:49:14.429500 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.429471    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db7b55f4-0dc4-46d1-b2ce-fb1d47895676" (UID: "db7b55f4-0dc4-46d1-b2ce-fb1d47895676"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:49:14.429708 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.429668    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db7b55f4-0dc4-46d1-b2ce-fb1d47895676" (UID: "db7b55f4-0dc4-46d1-b2ce-fb1d47895676"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:49:14.429854 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.429832    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-token" (OuterVolumeSpecName: "istio-token") pod "db7b55f4-0dc4-46d1-b2ce-fb1d47895676" (UID: "db7b55f4-0dc4-46d1-b2ce-fb1d47895676"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:49:14.429925 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.429868    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-kube-api-access-jjgfv" (OuterVolumeSpecName: "kube-api-access-jjgfv") pod "db7b55f4-0dc4-46d1-b2ce-fb1d47895676" (UID: "db7b55f4-0dc4-46d1-b2ce-fb1d47895676"). InnerVolumeSpecName "kube-api-access-jjgfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:49:14.528581 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.528548    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:49:14.528581 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.528578    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjgfv\" (UniqueName: \"kubernetes.io/projected/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-kube-api-access-jjgfv\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:49:14.528581 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.528590    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-data\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:49:14.528943 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.528600    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istiod-ca-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:49:14.528943 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.528608    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-envoy\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:49:14.528943 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.528616    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db7b55f4-0dc4-46d1-b2ce-fb1d47895676-istio-podinfo\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:49:14.681685 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.681593    2557 generic.go:356] "Generic (PLEG): container finished" podID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerID="cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846" exitCode=0
Jul 10 02:49:14.681685 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.681646    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"
Jul 10 02:49:14.681944 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.681678    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" event={"ID":"db7b55f4-0dc4-46d1-b2ce-fb1d47895676","Type":"ContainerDied","Data":"cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846"}
Jul 10 02:49:14.681944 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.681730    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn" event={"ID":"db7b55f4-0dc4-46d1-b2ce-fb1d47895676","Type":"ContainerDied","Data":"ed115a4749c2ca513861de50bc32183d49e4d99645d4da264ecc8db70ff7db56"}
Jul 10 02:49:14.681944 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.681754    2557 scope.go:117] "RemoveContainer" containerID="cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846"
Jul 10 02:49:14.690927 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.690904    2557 scope.go:117] "RemoveContainer" containerID="228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc"
Jul 10 02:49:14.699748 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.699728    2557 scope.go:117] "RemoveContainer" containerID="cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846"
Jul 10 02:49:14.700044 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:49:14.700019    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846\": container with ID starting with cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846 not found: ID does not exist" containerID="cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846"
Jul 10 02:49:14.700139 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.700049    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846"} err="failed to get container status \"cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846\": rpc error: code = NotFound desc = could not find container \"cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846\": container with ID starting with cbc227547c6d4eb468562a63b0e7ea62c8a17a912f14a5d0a11f0bfca8ae6846 not found: ID does not exist"
Jul 10 02:49:14.700139 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.700072    2557 scope.go:117] "RemoveContainer" containerID="228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc"
Jul 10 02:49:14.700334 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:49:14.700318    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc\": container with ID starting with 228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc not found: ID does not exist" containerID="228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc"
Jul 10 02:49:14.700374 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.700349    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc"} err="failed to get container status \"228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc\": rpc error: code = NotFound desc = could not find container \"228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc\": container with ID starting with 228a6304d4fdabcbb464e74bce444783f753894f4f39937a7160cd33aca967dc not found: ID does not exist"
Jul 10 02:49:14.703649 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.703629    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"]
Jul 10 02:49:14.712405 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:14.712383    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-20e0d1-openshift-default-778597f77c-v24rn"]
Jul 10 02:49:16.242221 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:16.242147    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" path="/var/lib/kubelet/pods/db7b55f4-0dc4-46d1-b2ce-fb1d47895676/volumes"
Jul 10 02:49:17.842945 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.842913    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"]
Jul 10 02:49:17.843328 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.843314    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy"
Jul 10 02:49:17.843374 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.843330    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy"
Jul 10 02:49:17.843374 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.843344    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy"
Jul 10 02:49:17.843374 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.843350    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy"
Jul 10 02:49:17.843461 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.843423    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy"
Jul 10 02:49:17.843461 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.843432    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="db7b55f4-0dc4-46d1-b2ce-fb1d47895676" containerName="istio-proxy"
Jul 10 02:49:17.847944 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.847921    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:17.849898 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.849865    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-de8279-openshift-default-dockercfg-dgfhs\""
Jul 10 02:49:17.858628 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.858605    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"]
Jul 10 02:49:17.958487 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.958449    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hsfv\" (UniqueName: \"kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-kube-api-access-4hsfv\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:17.958815 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.958780    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-token\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:17.958942 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.958858    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-credential-socket\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:17.958942 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.958920    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-podinfo\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:17.959075 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.958966    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-socket\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:17.959075 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.959023    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-envoy\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:17.959075 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.959069    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-certs\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:17.959243 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.959099    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-data\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:17.959243 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:17.959200    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istiod-ca-cert\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.060618 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.060581    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-envoy\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.060829 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.060630    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-certs\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.060829 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.060654    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-data\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.060829 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.060687    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istiod-ca-cert\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.060829 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.060711    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hsfv\" (UniqueName: \"kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-kube-api-access-4hsfv\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.060829 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.060740    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-token\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.061097 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.060915    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-credential-socket\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.061097 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.060989    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-podinfo\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.061097 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.061041    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-socket\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.061248 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.061093    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-certs\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.061248 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.061179    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-data\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.061365 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.061289    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-credential-socket\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.061481 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.061448    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-socket\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.061593 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.061455    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istiod-ca-cert\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.063187 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.063157    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-envoy\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.063372 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.063357    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-podinfo\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.072259 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.072237    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-token\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.072360 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.072317    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hsfv\" (UniqueName: \"kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-kube-api-access-4hsfv\") pod \"e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") " pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.159564 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.159488    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:18.298094 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.298053    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"]
Jul 10 02:49:18.299924 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:49:18.299872    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d5dd50_2892_4125_bc22_0ad25ca07bb7.slice/crio-392b9097aef41bea32442e3c8275588866c2775c2940fd1cd11b1e8e8321ba1c WatchSource:0}: Error finding container 392b9097aef41bea32442e3c8275588866c2775c2940fd1cd11b1e8e8321ba1c: Status 404 returned error can't find the container with id 392b9097aef41bea32442e3c8275588866c2775c2940fd1cd11b1e8e8321ba1c
Jul 10 02:49:18.302177 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.302150    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:18.302257 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.302211    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:18.302257 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.302241    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:18.704278 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.704242    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" event={"ID":"d2d5dd50-2892-4125-bc22-0ad25ca07bb7","Type":"ContainerStarted","Data":"b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812"}
Jul 10 02:49:18.704463 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.704302    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" event={"ID":"d2d5dd50-2892-4125-bc22-0ad25ca07bb7","Type":"ContainerStarted","Data":"392b9097aef41bea32442e3c8275588866c2775c2940fd1cd11b1e8e8321ba1c"}
Jul 10 02:49:18.724969 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:18.724920    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" podStartSLOduration=1.724903664 podStartE2EDuration="1.724903664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 02:49:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:49:18.723671699 +0000 UTC m=+2515.039981595" watchObservedRunningTime="2026-07-10 02:49:18.724903664 +0000 UTC m=+2515.041213535"
Jul 10 02:49:19.159994 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:19.159960    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:19.164418 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:19.164393    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:19.711888 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:19.711855    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:19.717320 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:19.717293    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:26.756859 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:26.756828    2557 generic.go:356] "Generic (PLEG): container finished" podID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerID="a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273" exitCode=0
Jul 10 02:49:26.757305 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:26.756905    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" event={"ID":"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697","Type":"ContainerDied","Data":"a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273"}
Jul 10 02:49:26.757305 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:26.757293    2557 scope.go:117] "RemoveContainer" containerID="a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273"
Jul 10 02:49:26.758246 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:26.758221    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:26.758307 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:26.758288    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:26.758343 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:26.758320    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:27.670788 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:27.670754    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:27.763063 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:27.762996    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" event={"ID":"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697","Type":"ContainerStarted","Data":"e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44"}
Jul 10 02:49:28.328611 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:28.328575    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:28.328996 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:28.328972    2557 patch_prober.go:28] interesting pod/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 10 02:49:28.329189 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:28.329161    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 10 02:49:29.328272 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:29.327821    2557 patch_prober.go:28] interesting pod/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 10 02:49:29.328272 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:29.327904    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 10 02:49:30.327981 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:30.327948    2557 patch_prober.go:28] interesting pod/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 10 02:49:30.328148 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:30.328020    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 10 02:49:31.328420 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:31.328377    2557 patch_prober.go:28] interesting pod/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body=
Jul 10 02:49:31.328848 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:31.328443    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused"
Jul 10 02:49:32.331911 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:32.331861    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:32.332319 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:32.332300    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:32.333053 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:32.333033    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:49:33.797708 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:33.797672    2557 generic.go:356] "Generic (PLEG): container finished" podID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerID="b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812" exitCode=0
Jul 10 02:49:33.798281 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:33.797746    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" event={"ID":"d2d5dd50-2892-4125-bc22-0ad25ca07bb7","Type":"ContainerDied","Data":"b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812"}
Jul 10 02:49:33.798359 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:33.798332    2557 scope.go:117] "RemoveContainer" containerID="b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812"
Jul 10 02:49:33.799368 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:33.799343    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:33.799444 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:33.799427    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:33.799491 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:33.799476    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 10 02:49:34.712076 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:34.712030    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:34.807323 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:34.807286    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" event={"ID":"d2d5dd50-2892-4125-bc22-0ad25ca07bb7","Type":"ContainerStarted","Data":"f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43"}
Jul 10 02:49:35.159962 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:35.159928    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:35.161084 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:35.161052    2557 patch_prober.go:28] interesting pod/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 10 02:49:35.161236 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:35.161122    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 10 02:49:36.160115 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:36.160081    2557 patch_prober.go:28] interesting pod/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 10 02:49:36.160501 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:36.160136    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 10 02:49:37.160712 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:37.160672    2557 patch_prober.go:28] interesting pod/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 10 02:49:37.161139 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:37.160733    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 10 02:49:38.160579 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:38.160541    2557 patch_prober.go:28] interesting pod/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 10 02:49:38.160766 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:38.160611    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 10 02:49:39.163949 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:39.163918    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:39.164347 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:39.164270    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:49:39.165011 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:49:39.164987    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:52:24.325936 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:52:24.325906    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:52:24.345226 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:52:24.345202    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:56:02.060927 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:02.060892    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7632-140000511603376"]
Jul 10 02:56:02.064355 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:02.064339    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7632-140000511603376"
Jul 10 02:56:02.066053 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:02.066035    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-hw75f\""
Jul 10 02:56:02.070140 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:02.070121    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7632-140000511603376"]
Jul 10 02:56:02.206468 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:02.206428    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrcn8\" (UniqueName: \"kubernetes.io/projected/e8043643-f0f8-4024-952c-536847c6657a-kube-api-access-rrcn8\") pod \"test-curl-7632-140000511603376\" (UID: \"e8043643-f0f8-4024-952c-536847c6657a\") " pod="openshift-ingress/test-curl-7632-140000511603376"
Jul 10 02:56:02.307118 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:02.307083    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrcn8\" (UniqueName: \"kubernetes.io/projected/e8043643-f0f8-4024-952c-536847c6657a-kube-api-access-rrcn8\") pod \"test-curl-7632-140000511603376\" (UID: \"e8043643-f0f8-4024-952c-536847c6657a\") " pod="openshift-ingress/test-curl-7632-140000511603376"
Jul 10 02:56:02.319167 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:02.319099    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrcn8\" (UniqueName: \"kubernetes.io/projected/e8043643-f0f8-4024-952c-536847c6657a-kube-api-access-rrcn8\") pod \"test-curl-7632-140000511603376\" (UID: \"e8043643-f0f8-4024-952c-536847c6657a\") " pod="openshift-ingress/test-curl-7632-140000511603376"
Jul 10 02:56:02.373526 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:02.373481    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7632-140000511603376"
Jul 10 02:56:02.506628 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:02.506605    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7632-140000511603376"]
Jul 10 02:56:02.508637 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:56:02.508600    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8043643_f0f8_4024_952c_536847c6657a.slice/crio-6cfb46dbe3bb8f87f0c711bebc29ebaff7530cb24cc09f3fec85f6dd9aeee163 WatchSource:0}: Error finding container 6cfb46dbe3bb8f87f0c711bebc29ebaff7530cb24cc09f3fec85f6dd9aeee163: Status 404 returned error can't find the container with id 6cfb46dbe3bb8f87f0c711bebc29ebaff7530cb24cc09f3fec85f6dd9aeee163
Jul 10 02:56:02.509988 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:02.509971    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 10 02:56:02.701320 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:02.701230    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7632-140000511603376" event={"ID":"e8043643-f0f8-4024-952c-536847c6657a","Type":"ContainerStarted","Data":"6cfb46dbe3bb8f87f0c711bebc29ebaff7530cb24cc09f3fec85f6dd9aeee163"}
Jul 10 02:56:03.708387 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:03.708351    2557 generic.go:356] "Generic (PLEG): container finished" podID="e8043643-f0f8-4024-952c-536847c6657a" containerID="101425be5ebdc3f8775b8de96dc0c04411cc751cfbe392587242ee37362c1249" exitCode=0
Jul 10 02:56:03.708883 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:03.708410    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7632-140000511603376" event={"ID":"e8043643-f0f8-4024-952c-536847c6657a","Type":"ContainerDied","Data":"101425be5ebdc3f8775b8de96dc0c04411cc751cfbe392587242ee37362c1249"}
Jul 10 02:56:04.861320 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:04.861290    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7632-140000511603376"
Jul 10 02:56:04.895022 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:04.894984    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7632-140000511603376_e8043643-f0f8-4024-952c-536847c6657a/test-curl-7632-140000511603376/0.log"
Jul 10 02:56:04.923889 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:04.923862    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7632-140000511603376"]
Jul 10 02:56:04.928458 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:04.928434    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7632-140000511603376"]
Jul 10 02:56:04.930004 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:04.929977    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrcn8\" (UniqueName: \"kubernetes.io/projected/e8043643-f0f8-4024-952c-536847c6657a-kube-api-access-rrcn8\") pod \"e8043643-f0f8-4024-952c-536847c6657a\" (UID: \"e8043643-f0f8-4024-952c-536847c6657a\") "
Jul 10 02:56:04.932264 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:04.932244    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8043643-f0f8-4024-952c-536847c6657a-kube-api-access-rrcn8" (OuterVolumeSpecName: "kube-api-access-rrcn8") pod "e8043643-f0f8-4024-952c-536847c6657a" (UID: "e8043643-f0f8-4024-952c-536847c6657a"). InnerVolumeSpecName "kube-api-access-rrcn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:56:05.030799 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:05.030769    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrcn8\" (UniqueName: \"kubernetes.io/projected/e8043643-f0f8-4024-952c-536847c6657a-kube-api-access-rrcn8\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:56:05.720031 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:05.719999    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7632-140000511603376"
Jul 10 02:56:05.720201 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:05.720037    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6cfb46dbe3bb8f87f0c711bebc29ebaff7530cb24cc09f3fec85f6dd9aeee163"
Jul 10 02:56:05.729043 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:56:05.729013    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7632-140000511603376\" is forbidden: User \"system:node:ip-10-0-131-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-213.ec2.internal' and this object" podUID="e8043643-f0f8-4024-952c-536847c6657a" pod="openshift-ingress/test-curl-7632-140000511603376"
Jul 10 02:56:06.242214 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:06.242179    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8043643-f0f8-4024-952c-536847c6657a" path="/var/lib/kubelet/pods/e8043643-f0f8-4024-952c-536847c6657a/volumes"
Jul 10 02:56:08.083402 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.083368    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7632-140000509483312"]
Jul 10 02:56:08.083785 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.083732    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8043643-f0f8-4024-952c-536847c6657a" containerName="test-curl-7632-140000511603376"
Jul 10 02:56:08.083785 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.083743    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8043643-f0f8-4024-952c-536847c6657a" containerName="test-curl-7632-140000511603376"
Jul 10 02:56:08.083888 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.083831    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8043643-f0f8-4024-952c-536847c6657a" containerName="test-curl-7632-140000511603376"
Jul 10 02:56:08.086938 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.086919    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7632-140000509483312"
Jul 10 02:56:08.088598 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.088582    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-hw75f\""
Jul 10 02:56:08.092839 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.092790    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7632-140000509483312"]
Jul 10 02:56:08.161933 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.161901    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2czs\" (UniqueName: \"kubernetes.io/projected/ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac-kube-api-access-v2czs\") pod \"test-curl-7632-140000509483312\" (UID: \"ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac\") " pod="openshift-ingress/test-curl-7632-140000509483312"
Jul 10 02:56:08.262600 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.262564    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2czs\" (UniqueName: \"kubernetes.io/projected/ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac-kube-api-access-v2czs\") pod \"test-curl-7632-140000509483312\" (UID: \"ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac\") " pod="openshift-ingress/test-curl-7632-140000509483312"
Jul 10 02:56:08.270537 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.270511    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2czs\" (UniqueName: \"kubernetes.io/projected/ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac-kube-api-access-v2czs\") pod \"test-curl-7632-140000509483312\" (UID: \"ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac\") " pod="openshift-ingress/test-curl-7632-140000509483312"
Jul 10 02:56:08.396972 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.396889    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7632-140000509483312"
Jul 10 02:56:08.528641 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.528621    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7632-140000509483312"]
Jul 10 02:56:08.531070 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:56:08.531047    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee9e9a0e_6388_4ddc_90b9_a1d9769c76ac.slice/crio-377c6a562f5bcf575c4ec06177bf6f8e749271e707b26ca40ae5359f7eb2bd0b WatchSource:0}: Error finding container 377c6a562f5bcf575c4ec06177bf6f8e749271e707b26ca40ae5359f7eb2bd0b: Status 404 returned error can't find the container with id 377c6a562f5bcf575c4ec06177bf6f8e749271e707b26ca40ae5359f7eb2bd0b
Jul 10 02:56:08.736127 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:08.736040    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7632-140000509483312" event={"ID":"ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac","Type":"ContainerStarted","Data":"377c6a562f5bcf575c4ec06177bf6f8e749271e707b26ca40ae5359f7eb2bd0b"}
Jul 10 02:56:09.742516 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:09.742433    2557 generic.go:356] "Generic (PLEG): container finished" podID="ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac" containerID="897993fa08b23ca002fe1b994c715ba2fb4b046de2c720cdb5a9d3fe4c047297" exitCode=0
Jul 10 02:56:09.742516 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:09.742488    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7632-140000509483312" event={"ID":"ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac","Type":"ContainerDied","Data":"897993fa08b23ca002fe1b994c715ba2fb4b046de2c720cdb5a9d3fe4c047297"}
Jul 10 02:56:10.881722 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:10.881695    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7632-140000509483312"
Jul 10 02:56:10.902366 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:10.902327    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7632-140000509483312_ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac/test-curl-7632-140000509483312/0.log"
Jul 10 02:56:10.928772 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:10.928746    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7632-140000509483312"]
Jul 10 02:56:10.937026 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:10.937000    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7632-140000509483312"]
Jul 10 02:56:10.988563 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:10.988526    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2czs\" (UniqueName: \"kubernetes.io/projected/ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac-kube-api-access-v2czs\") pod \"ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac\" (UID: \"ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac\") "
Jul 10 02:56:10.990536 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:10.990513    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac-kube-api-access-v2czs" (OuterVolumeSpecName: "kube-api-access-v2czs") pod "ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac" (UID: "ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac"). InnerVolumeSpecName "kube-api-access-v2czs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:56:11.089971 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:11.089932    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2czs\" (UniqueName: \"kubernetes.io/projected/ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac-kube-api-access-v2czs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:56:11.753059 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:11.753032    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="377c6a562f5bcf575c4ec06177bf6f8e749271e707b26ca40ae5359f7eb2bd0b"
Jul 10 02:56:11.753230 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:11.753068    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7632-140000509483312"
Jul 10 02:56:11.762142 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:56:11.762117    2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7632-140000509483312\" is forbidden: User \"system:node:ip-10-0-131-213.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-131-213.ec2.internal' and this object" podUID="ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac" pod="openshift-ingress/test-curl-7632-140000509483312"
Jul 10 02:56:12.241817 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:12.241773    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac" path="/var/lib/kubelet/pods/ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac/volumes"
Jul 10 02:56:43.076204 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:43.076161    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"]
Jul 10 02:56:43.076851 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:43.076418    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy" containerID="cri-o://e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44" gracePeriod=30
Jul 10 02:56:47.334777 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:47.334691    2557 patch_prober.go:28] interesting pod/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 10 02:56:47.334777 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:47.334767    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 10 02:56:48.335958 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.335935    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:56:48.390763 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.390694    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-envoy\") pod \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") "
Jul 10 02:56:48.390763 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.390730    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-data\") pod \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") "
Jul 10 02:56:48.390998 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.390769    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-certs\") pod \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") "
Jul 10 02:56:48.390998 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.390845    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-podinfo\") pod \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") "
Jul 10 02:56:48.390998 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.390891    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m65p2\" (UniqueName: \"kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-kube-api-access-m65p2\") pod \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") "
Jul 10 02:56:48.390998 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.390930    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-socket\") pod \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") "
Jul 10 02:56:48.390998 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.390961    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-credential-socket\") pod \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") "
Jul 10 02:56:48.390998 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.390983    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-token\") pod \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") "
Jul 10 02:56:48.391309 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.391022    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istiod-ca-cert\") pod \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\" (UID: \"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697\") "
Jul 10 02:56:48.391309 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.391111    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" (UID: "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:56:48.391428 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.391337    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" (UID: "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:56:48.391428 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.391377    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:56:48.391428 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.391415    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" (UID: "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:56:48.391579 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.391557    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" (UID: "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:56:48.391644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.391577    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-data" (OuterVolumeSpecName: "istio-data") pod "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" (UID: "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:56:48.393143 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.393117    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" (UID: "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:56:48.393322 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.393304    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-token" (OuterVolumeSpecName: "istio-token") pod "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" (UID: "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:56:48.393544 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.393515    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-kube-api-access-m65p2" (OuterVolumeSpecName: "kube-api-access-m65p2") pod "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" (UID: "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697"). InnerVolumeSpecName "kube-api-access-m65p2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:56:48.393633 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.393552    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" (UID: "7cc8bdf6-924e-4fa4-a7a8-982ae5caf697"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:56:48.491899 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.491863    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-envoy\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:56:48.491899 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.491894    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-data\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:56:48.491899 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.491907    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-podinfo\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:56:48.492152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.491918    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m65p2\" (UniqueName: \"kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-kube-api-access-m65p2\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:56:48.492152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.491932    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-workload-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:56:48.492152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.491944    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-credential-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:56:48.492152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.491954    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:56:48.492152 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.491967    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697-istiod-ca-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:56:48.930716 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.930683    2557 generic.go:356] "Generic (PLEG): container finished" podID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerID="e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44" exitCode=0
Jul 10 02:56:48.930930 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.930735    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"
Jul 10 02:56:48.930930 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.930773    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" event={"ID":"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697","Type":"ContainerDied","Data":"e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44"}
Jul 10 02:56:48.930930 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.930848    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9" event={"ID":"7cc8bdf6-924e-4fa4-a7a8-982ae5caf697","Type":"ContainerDied","Data":"821cf3bb4b5faec093b78b4f012354c3848b17bd11ad3d3a5433cd6ada290058"}
Jul 10 02:56:48.930930 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.930881    2557 scope.go:117] "RemoveContainer" containerID="e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44"
Jul 10 02:56:48.941902 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.941875    2557 scope.go:117] "RemoveContainer" containerID="a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273"
Jul 10 02:56:48.949944 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.949926    2557 scope.go:117] "RemoveContainer" containerID="e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44"
Jul 10 02:56:48.950196 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:56:48.950171    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44\": container with ID starting with e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44 not found: ID does not exist" containerID="e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44"
Jul 10 02:56:48.950265 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.950203    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44"} err="failed to get container status \"e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44\": rpc error: code = NotFound desc = could not find container \"e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44\": container with ID starting with e3afa737685e6ca7c142a29bf0d2aeacfc991347bd8b1b4f7ac855f200b18f44 not found: ID does not exist"
Jul 10 02:56:48.950265 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.950220    2557 scope.go:117] "RemoveContainer" containerID="a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273"
Jul 10 02:56:48.950444 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:56:48.950428    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273\": container with ID starting with a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273 not found: ID does not exist" containerID="a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273"
Jul 10 02:56:48.950482 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.950446    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273"} err="failed to get container status \"a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273\": rpc error: code = NotFound desc = could not find container \"a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273\": container with ID starting with a72b07e862ae3e420e5087f7419fe444fb8f12758955277a54d842896d732273 not found: ID does not exist"
Jul 10 02:56:48.955654 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.955631    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"]
Jul 10 02:56:48.959597 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:48.959579    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-d5b98a-openshift-default-6cc7dbf88c-ks9s9"]
Jul 10 02:56:50.242396 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:56:50.242363    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" path="/var/lib/kubelet/pods/7cc8bdf6-924e-4fa4-a7a8-982ae5caf697/volumes"
Jul 10 02:57:09.264761 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:09.264719    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"]
Jul 10 02:57:09.265248 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:09.264947    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerName="istio-proxy" containerID="cri-o://f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43" gracePeriod=30
Jul 10 02:57:09.974166 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:09.974134    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b55d68598-pdhkg_751cb34d-d874-47ab-ae3c-883d57c5d863/authorino/0.log"
Jul 10 02:57:14.510097 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.510070    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:57:14.628079 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.627990    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4hsfv\" (UniqueName: \"kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-kube-api-access-4hsfv\") pod \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") "
Jul 10 02:57:14.628079 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628029    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-podinfo\") pod \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") "
Jul 10 02:57:14.628271 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628084    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-envoy\") pod \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") "
Jul 10 02:57:14.628271 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628111    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-socket\") pod \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") "
Jul 10 02:57:14.628271 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628172    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-data\") pod \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") "
Jul 10 02:57:14.628271 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628197    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-certs\") pod \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") "
Jul 10 02:57:14.628271 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628224    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-credential-socket\") pod \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") "
Jul 10 02:57:14.628536 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628297    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istiod-ca-cert\") pod \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") "
Jul 10 02:57:14.628536 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628338    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-token\") pod \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\" (UID: \"d2d5dd50-2892-4125-bc22-0ad25ca07bb7\") "
Jul 10 02:57:14.628644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628510    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d2d5dd50-2892-4125-bc22-0ad25ca07bb7" (UID: "d2d5dd50-2892-4125-bc22-0ad25ca07bb7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:57:14.628644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628558    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d2d5dd50-2892-4125-bc22-0ad25ca07bb7" (UID: "d2d5dd50-2892-4125-bc22-0ad25ca07bb7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:57:14.628644 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628587    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d2d5dd50-2892-4125-bc22-0ad25ca07bb7" (UID: "d2d5dd50-2892-4125-bc22-0ad25ca07bb7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:57:14.628842 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628675    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-certs\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:57:14.628842 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628695    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-workload-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:57:14.628955 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.628897    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d2d5dd50-2892-4125-bc22-0ad25ca07bb7" (UID: "d2d5dd50-2892-4125-bc22-0ad25ca07bb7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 10 02:57:14.629081 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.629053    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-data" (OuterVolumeSpecName: "istio-data") pod "d2d5dd50-2892-4125-bc22-0ad25ca07bb7" (UID: "d2d5dd50-2892-4125-bc22-0ad25ca07bb7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:57:14.630439 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.630410    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d2d5dd50-2892-4125-bc22-0ad25ca07bb7" (UID: "d2d5dd50-2892-4125-bc22-0ad25ca07bb7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 10 02:57:14.630439 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.630418    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d2d5dd50-2892-4125-bc22-0ad25ca07bb7" (UID: "d2d5dd50-2892-4125-bc22-0ad25ca07bb7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 10 02:57:14.630732 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.630713    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-token" (OuterVolumeSpecName: "istio-token") pod "d2d5dd50-2892-4125-bc22-0ad25ca07bb7" (UID: "d2d5dd50-2892-4125-bc22-0ad25ca07bb7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:57:14.630732 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.630725    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-kube-api-access-4hsfv" (OuterVolumeSpecName: "kube-api-access-4hsfv") pod "d2d5dd50-2892-4125-bc22-0ad25ca07bb7" (UID: "d2d5dd50-2892-4125-bc22-0ad25ca07bb7"). InnerVolumeSpecName "kube-api-access-4hsfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 10 02:57:14.729667 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.729630    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-data\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:57:14.729667 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.729662    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-credential-socket\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:57:14.729667 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.729673    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istiod-ca-cert\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:57:14.729945 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.729682    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-token\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:57:14.729945 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.729690    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4hsfv\" (UniqueName: \"kubernetes.io/projected/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-kube-api-access-4hsfv\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:57:14.729945 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.729701    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-podinfo\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:57:14.729945 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:14.729709    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2d5dd50-2892-4125-bc22-0ad25ca07bb7-istio-envoy\") on node \"ip-10-0-131-213.ec2.internal\" DevicePath \"\""
Jul 10 02:57:15.059279 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.059246    2557 generic.go:356] "Generic (PLEG): container finished" podID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerID="f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43" exitCode=0
Jul 10 02:57:15.059462 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.059298    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"
Jul 10 02:57:15.059462 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.059330    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" event={"ID":"d2d5dd50-2892-4125-bc22-0ad25ca07bb7","Type":"ContainerDied","Data":"f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43"}
Jul 10 02:57:15.059462 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.059393    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2" event={"ID":"d2d5dd50-2892-4125-bc22-0ad25ca07bb7","Type":"ContainerDied","Data":"392b9097aef41bea32442e3c8275588866c2775c2940fd1cd11b1e8e8321ba1c"}
Jul 10 02:57:15.059462 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.059424    2557 scope.go:117] "RemoveContainer" containerID="f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43"
Jul 10 02:57:15.068627 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.068606    2557 scope.go:117] "RemoveContainer" containerID="b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812"
Jul 10 02:57:15.076595 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.076577    2557 scope.go:117] "RemoveContainer" containerID="f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43"
Jul 10 02:57:15.076865 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:57:15.076845    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43\": container with ID starting with f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43 not found: ID does not exist" containerID="f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43"
Jul 10 02:57:15.076909 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.076874    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43"} err="failed to get container status \"f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43\": rpc error: code = NotFound desc = could not find container \"f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43\": container with ID starting with f8ea65d44159ae847540cfcd34f3e81e2835fd242a508e9d4520007909312d43 not found: ID does not exist"
Jul 10 02:57:15.076909 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.076894    2557 scope.go:117] "RemoveContainer" containerID="b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812"
Jul 10 02:57:15.077108 ip-10-0-131-213 kubenswrapper[2557]: E0710 02:57:15.077089    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812\": container with ID starting with b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812 not found: ID does not exist" containerID="b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812"
Jul 10 02:57:15.077145 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.077115    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812"} err="failed to get container status \"b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812\": rpc error: code = NotFound desc = could not find container \"b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812\": container with ID starting with b079456456643318afc77a3725ad4827466cbef05940ed8cd7fc6736deb6d812 not found: ID does not exist"
Jul 10 02:57:15.081732 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.081709    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"]
Jul 10 02:57:15.086542 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.086517    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-de8279-openshift-default-57dd96c74c-rcrm2"]
Jul 10 02:57:15.126015 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.125991    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4z6n8_bd63e164-0a54-474a-b508-978d78963df3/manager/0.log"
Jul 10 02:57:15.254130 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.254102    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nt97c_9dfe9a84-a887-400c-9cd1-ebb4027492c4/manager/0.log"
Jul 10 02:57:15.386958 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.386878    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56f677784f-5grmt_05b08f09-b1d2-44b4-9ba2-691b72cbbc1a/manager/0.log"
Jul 10 02:57:15.507709 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.507684    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hqtwr_217f9b2e-454b-4739-9d19-155d79101883/server/0.log"
Jul 10 02:57:15.632880 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.632850    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2b76z_c4af3523-66b5-4c1e-a6e9-4292aeb12df0/manager/0.log"
Jul 10 02:57:15.750913 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.750831    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b9cbb96-6f4c6_d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9/copy-manifests/0.log"
Jul 10 02:57:15.758755 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:15.758724    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b9cbb96-6f4c6_d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9/manager/0.log"
Jul 10 02:57:16.245009 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:16.242898    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" path="/var/lib/kubelet/pods/d2d5dd50-2892-4125-bc22-0ad25ca07bb7/volumes"
Jul 10 02:57:16.432453 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:16.432424    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-654967854b-8p7lc_1c1df2d1-4327-40cf-815c-e0c07946e15f/maas-api/0.log"
Jul 10 02:57:18.104123 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:18.104096    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-sz4gr_b1d34833-be71-4e9f-8873-93a8aee9baf3/postgres/0.log"
Jul 10 02:57:19.481155 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:19.481124    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b55d68598-pdhkg_751cb34d-d874-47ab-ae3c-883d57c5d863/authorino/0.log"
Jul 10 02:57:19.609474 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:19.609430    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-m94kz_abaf9abf-1334-4a5f-a143-961d7da59892/manager/0.log"
Jul 10 02:57:19.969054 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:19.969015    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hjwmj_7d4ae36e-5089-43b4-acd9-d9ce955e26bf/registry-server/0.log"
Jul 10 02:57:20.105075 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:20.105042    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-t7rtp_0a5143d3-e139-4f28-8619-f9d5e2980a8f/manager/0.log"
Jul 10 02:57:20.739180 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:20.739145    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46s2l9_6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec/istio-proxy/0.log"
Jul 10 02:57:20.983357 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:20.983328    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cf46c8b8c-7gbvk_2cf43624-9057-462b-ba5e-e3a35ae43ec6/kube-auth-proxy/0.log"
Jul 10 02:57:21.234449 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:21.234422    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xngqk_964c11a5-ebcf-40b5-8ff5-9b8156cc6c24/istio-proxy/1.log"
Jul 10 02:57:21.366282 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:21.366250    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-78tkn_37f57f08-7bf6-4b56-abf5-a9526f547379/payload-pre-processing/0.log"
Jul 10 02:57:21.498905 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:21.498822    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-z6jdx_7d0aa707-2ba1-4753-9b07-c14793ab2861/payload-processing/0.log"
Jul 10 02:57:22.108426 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:22.108395    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-r5sf6_269c6d58-4eaa-4f66-bdf1-fd697e7506b5/main/0.log"
Jul 10 02:57:22.600398 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:22.600371    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-x5bdr_aee94787-f990-4cc9-a3c0-e0efbe2e1ccc/main/0.log"
Jul 10 02:57:24.353557 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:24.353530    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:57:24.372521 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:24.372492    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-r4vzs_cbeea170-d687-476c-b932-01800b00ecae/ovn-acl-logging/0.log"
Jul 10 02:57:30.033304 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:30.033272    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mb7lt_796726ff-5fe1-48dd-9ea0-e5fae11d44e2/global-pull-secret-syncer/0.log"
Jul 10 02:57:30.246103 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:30.246072    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p7ftq_b7ae93db-a16c-4e3b-bb52-59ccc91e033a/konnectivity-agent/0.log"
Jul 10 02:57:30.293869 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:30.293766    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-213.ec2.internal_d0d70d9ea477f5082aeebc1d283121de/haproxy/0.log"
Jul 10 02:57:35.251871 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:35.251837    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b55d68598-pdhkg_751cb34d-d874-47ab-ae3c-883d57c5d863/authorino/0.log"
Jul 10 02:57:35.283627 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:35.283594    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-m94kz_abaf9abf-1334-4a5f-a143-961d7da59892/manager/0.log"
Jul 10 02:57:35.418069 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:35.418039    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hjwmj_7d4ae36e-5089-43b4-acd9-d9ce955e26bf/registry-server/0.log"
Jul 10 02:57:35.615504 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:35.615471    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-t7rtp_0a5143d3-e139-4f28-8619-f9d5e2980a8f/manager/0.log"
Jul 10 02:57:37.817676 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:37.817646    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-765lk_5a4e5f18-4506-4200-ad5b-44b0d14b7337/node-exporter/0.log"
Jul 10 02:57:37.840215 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:37.840190    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-765lk_5a4e5f18-4506-4200-ad5b-44b0d14b7337/kube-rbac-proxy/0.log"
Jul 10 02:57:37.866635 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:37.866608    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-765lk_5a4e5f18-4506-4200-ad5b-44b0d14b7337/init-textfile/0.log"
Jul 10 02:57:39.648374 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648344    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"]
Jul 10 02:57:39.648749 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648676    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerName="istio-proxy"
Jul 10 02:57:39.648749 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648687    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerName="istio-proxy"
Jul 10 02:57:39.648749 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648708    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerName="istio-proxy"
Jul 10 02:57:39.648749 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648714    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerName="istio-proxy"
Jul 10 02:57:39.648749 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648722    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy"
Jul 10 02:57:39.648749 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648727    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy"
Jul 10 02:57:39.648749 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648733    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac" containerName="test-curl-7632-140000509483312"
Jul 10 02:57:39.648749 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648739    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac" containerName="test-curl-7632-140000509483312"
Jul 10 02:57:39.648749 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648747    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy"
Jul 10 02:57:39.648749 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648752    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy"
Jul 10 02:57:39.649092 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648838    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy"
Jul 10 02:57:39.649092 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648846    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cc8bdf6-924e-4fa4-a7a8-982ae5caf697" containerName="istio-proxy"
Jul 10 02:57:39.649092 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648853    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerName="istio-proxy"
Jul 10 02:57:39.649092 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648859    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee9e9a0e-6388-4ddc-90b9-a1d9769c76ac" containerName="test-curl-7632-140000509483312"
Jul 10 02:57:39.649092 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.648866    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2d5dd50-2892-4125-bc22-0ad25ca07bb7" containerName="istio-proxy"
Jul 10 02:57:39.651783 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.651763    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.653538 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.653515    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s4bld\"/\"openshift-service-ca.crt\""
Jul 10 02:57:39.653648 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.653575    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s4bld\"/\"default-dockercfg-q2jnq\""
Jul 10 02:57:39.653909 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.653893    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s4bld\"/\"kube-root-ca.crt\""
Jul 10 02:57:39.662580 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.662557    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"]
Jul 10 02:57:39.746491 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.746455    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-podres\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.746656 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.746520    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q749\" (UniqueName: \"kubernetes.io/projected/e82295a3-b524-4370-849e-206235263742-kube-api-access-2q749\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.746656 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.746547    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-lib-modules\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.746656 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.746563    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-proc\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.746656 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.746596    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-sys\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.847784 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.847737    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-sys\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.848011 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.847833    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-podres\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.848011 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.847867    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q749\" (UniqueName: \"kubernetes.io/projected/e82295a3-b524-4370-849e-206235263742-kube-api-access-2q749\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.848011 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.847889    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-lib-modules\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.848011 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.847902    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-sys\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.848011 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.847913    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-proc\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.848011 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.847963    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-proc\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.848011 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.847992    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-podres\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.848278 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.848019    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e82295a3-b524-4370-849e-206235263742-lib-modules\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.856472 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.856447    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q749\" (UniqueName: \"kubernetes.io/projected/e82295a3-b524-4370-849e-206235263742-kube-api-access-2q749\") pod \"perf-node-gather-daemonset-9pq7b\" (UID: \"e82295a3-b524-4370-849e-206235263742\") " pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:39.858766 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.858747    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-rr7f6_109d498f-fabd-4b87-b893-19dc87cb1f23/networking-console-plugin/0.log"
Jul 10 02:57:39.963737 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:39.963657    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:40.097754 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:40.097731    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"]
Jul 10 02:57:40.099794 ip-10-0-131-213 kubenswrapper[2557]: W0710 02:57:40.099769    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode82295a3_b524_4370_849e_206235263742.slice/crio-ede0a89e70c2da6ed41b65042f5f7940cdbd28fa2ca0997526d78f8ffc1c97cd WatchSource:0}: Error finding container ede0a89e70c2da6ed41b65042f5f7940cdbd28fa2ca0997526d78f8ffc1c97cd: Status 404 returned error can't find the container with id ede0a89e70c2da6ed41b65042f5f7940cdbd28fa2ca0997526d78f8ffc1c97cd
Jul 10 02:57:40.187064 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:40.187029    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b" event={"ID":"e82295a3-b524-4370-849e-206235263742","Type":"ContainerStarted","Data":"ede0a89e70c2da6ed41b65042f5f7940cdbd28fa2ca0997526d78f8ffc1c97cd"}
Jul 10 02:57:41.193329 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:41.193294    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b" event={"ID":"e82295a3-b524-4370-849e-206235263742","Type":"ContainerStarted","Data":"98a4670da11f108083d26c8b16b92c49b3f6902c0fbcc0ff923f3331bf52f708"}
Jul 10 02:57:41.193688 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:41.193356    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:41.209172 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:41.209126    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b" podStartSLOduration=2.209111541 podStartE2EDuration="2.209111541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 02:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 02:57:41.208356355 +0000 UTC m=+3017.524666237" watchObservedRunningTime="2026-07-10 02:57:41.209111541 +0000 UTC m=+3017.525421412"
Jul 10 02:57:42.921384 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:42.921352    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vsgwk_d480e3b6-008b-433b-8a35-6d023ed58eff/dns/0.log"
Jul 10 02:57:42.945197 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:42.945169    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vsgwk_d480e3b6-008b-433b-8a35-6d023ed58eff/kube-rbac-proxy/0.log"
Jul 10 02:57:42.969056 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:42.969019    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4gmwh_98184b4a-870b-47be-a682-e1f7d43effb0/dns-node-resolver/0.log"
Jul 10 02:57:43.713456 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:43.713424    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8w4fn_6e7c563a-d986-46de-994c-230d665e028b/node-ca/0.log"
Jul 10 02:57:44.733030 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:44.733002    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46s2l9_6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec/istio-proxy/0.log"
Jul 10 02:57:44.869814 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:44.869784    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cf46c8b8c-7gbvk_2cf43624-9057-462b-ba5e-e3a35ae43ec6/kube-auth-proxy/0.log"
Jul 10 02:57:44.932387 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:44.932358    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xngqk_964c11a5-ebcf-40b5-8ff5-9b8156cc6c24/istio-proxy/0.log"
Jul 10 02:57:45.002273 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:45.002165    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xngqk_964c11a5-ebcf-40b5-8ff5-9b8156cc6c24/istio-proxy/1.log"
Jul 10 02:57:45.042890 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:45.042857    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-78tkn_37f57f08-7bf6-4b56-abf5-a9526f547379/payload-pre-processing/0.log"
Jul 10 02:57:45.085039 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:45.085003    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-z6jdx_7d0aa707-2ba1-4753-9b07-c14793ab2861/payload-processing/0.log"
Jul 10 02:57:45.896212 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:45.896174    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rv9wg_d86f5caf-4a63-4947-b175-ae4a035dd52b/serve-healthcheck-canary/0.log"
Jul 10 02:57:46.584244 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:46.584213    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zccx2_0a41b0c6-0c4e-4605-b94f-3672543b59b3/kube-rbac-proxy/0.log"
Jul 10 02:57:46.608737 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:46.608703    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zccx2_0a41b0c6-0c4e-4605-b94f-3672543b59b3/exporter/0.log"
Jul 10 02:57:46.632550 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:46.632520    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zccx2_0a41b0c6-0c4e-4605-b94f-3672543b59b3/extractor/0.log"
Jul 10 02:57:47.207298 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:47.207269    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s4bld/perf-node-gather-daemonset-9pq7b"
Jul 10 02:57:48.962388 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:48.962358    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-4z6n8_bd63e164-0a54-474a-b508-978d78963df3/manager/0.log"
Jul 10 02:57:49.222119 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:49.222089    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-nt97c_9dfe9a84-a887-400c-9cd1-ebb4027492c4/manager/0.log"
Jul 10 02:57:49.400897 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:49.400862    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56f677784f-5grmt_05b08f09-b1d2-44b4-9ba2-691b72cbbc1a/manager/0.log"
Jul 10 02:57:49.427599 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:49.427528    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-hqtwr_217f9b2e-454b-4739-9d19-155d79101883/server/0.log"
Jul 10 02:57:49.537496 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:49.537466    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2b76z_c4af3523-66b5-4c1e-a6e9-4292aeb12df0/manager/0.log"
Jul 10 02:57:49.566923 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:49.566891    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b9cbb96-6f4c6_d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9/manager/0.log"
Jul 10 02:57:49.590275 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:49.590251    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78b9cbb96-6f4c6_d70fd7b3-cbbb-4a89-8a58-576aa6ae40b9/copy-manifests/0.log"
Jul 10 02:57:50.131493 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:50.131467    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46s2l9_6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec/istio-proxy/0.log"
Jul 10 02:57:50.252325 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:50.252294    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cf46c8b8c-7gbvk_2cf43624-9057-462b-ba5e-e3a35ae43ec6/kube-auth-proxy/0.log"
Jul 10 02:57:50.295554 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:50.295527    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xngqk_964c11a5-ebcf-40b5-8ff5-9b8156cc6c24/istio-proxy/0.log"
Jul 10 02:57:50.352548 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:50.352513    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xngqk_964c11a5-ebcf-40b5-8ff5-9b8156cc6c24/istio-proxy/1.log"
Jul 10 02:57:50.381153 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:50.381122    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-78tkn_37f57f08-7bf6-4b56-abf5-a9526f547379/payload-pre-processing/0.log"
Jul 10 02:57:50.413893 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:50.413796    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-z6jdx_7d0aa707-2ba1-4753-9b07-c14793ab2861/payload-processing/0.log"
Jul 10 02:57:51.048820 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:51.048783    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2dn8p_6f28baf9-70d7-4078-8fa4-6d7dd881321b/openshift-lws-operator/0.log"
Jul 10 02:57:51.711875 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:51.711844    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46s2l9_6e97fc62-5e41-4a29-bdfb-2af9d5e8e8ec/istio-proxy/0.log"
Jul 10 02:57:51.832502 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:51.832468    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cf46c8b8c-7gbvk_2cf43624-9057-462b-ba5e-e3a35ae43ec6/kube-auth-proxy/0.log"
Jul 10 02:57:51.874697 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:51.874670    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xngqk_964c11a5-ebcf-40b5-8ff5-9b8156cc6c24/istio-proxy/0.log"
Jul 10 02:57:51.935338 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:51.935303    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xngqk_964c11a5-ebcf-40b5-8ff5-9b8156cc6c24/istio-proxy/1.log"
Jul 10 02:57:51.965895 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:51.965793    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-78tkn_37f57f08-7bf6-4b56-abf5-a9526f547379/payload-pre-processing/0.log"
Jul 10 02:57:52.002540 ip-10-0-131-213 kubenswrapper[2557]: I0710 02:57:52.002512    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-z6jdx_7d0aa707-2ba1-4753-9b07-c14793ab2861/payload-processing/0.log"